Mark 2:16 & 17
"When the teachers of the law who were Pharisees saw him eating with the 'sinners' and tax collectors, they asked his disciples: 'Why does he eat with tax collectors and "sinners"?' On hearing this, Jesus said to them, 'It is not the healthy who need a doctor, but the sick. I have not come to call the righteous, but sinners.'" NIV translation
Not only did Levi feed the people that were following Jesus but there were also religious leaders there. Although they were probably not eating, these leaders were welcomed into Levi's home.
These men would have hated Levi and looked down on him because of his job but still they were allowed in and they spoke with Jesus' disciples.
These cowards did not have the courage to ask Jesus the question but instead went behind His back to ask His disciples why He was hanging out with "sinners". We should note that, if we are following Jesus, there will be cowardly gossips who talk about us behind our backs because they do not have the courage to face the Word of God.
Jesus heard about their question and gave a simple answer about His purpose on this earth. Just like the physically sick go to a doctor to be healed, He is the spiritual doctor and He has come to heal those who acknowledge that they are spiritually sick. When He says that He did not come to call the righteous, does that mean that He did not come for all men? No, it simply means that no man is righteous without Him even though these leaders were so full of themselves and their piety.