Jeremiah 5:1 & 2
"'Go up and down the streets of Jerusalem, look around and consider, search through her squares. If you can find but one person who deals honestly and seeks the truth, I will forgive this city. Although they say, "As surely as the Lord lives," still they are swearing falsely.'" NIV translation
A challenge is issued to find one righteous man in the whole city of Jerusalem. God offers to spare the entire city for one man but there is not even one. We remember, from Genesis 18, how Abraham had asked God to spare Sodom and Gomorrah if ten righteous men could be found. Either Abraham did not understand the depth of their sin or the heights of God's mercy when he stopped asking at ten.
We see that, in Jerusalem, there were those who were claiming to acknowledge God but it was not a reality in their lives. Jesus also said that there would be those that, on the last day, do not enter heaven even though they had done things in His name. We see from both cases that what matters is what is in the heart.