"So he got up, took the child and his mother and went to the land of Israel. But when he heard that Archelaus was reigning in Judea in place of his father Herod, he was afraid to go there. Having been warned in a dream, he withdrew to the district of Galilee, and he went and lived in a town called Nazareth. So was fulfilled what was said through the prophets, that he would be called a Nazarene." NIV translation
We see from this passage that Joseph and Mary knew that Jesus was the promised King of the Jews. Naturally, the King of the Jews would live in Jerusalem but, once again, God intervened and they went back to live in Nazareth which is where they were from.
In Isaiah 11:1, it was said that a shoot would come up from Jesse (the father of King David) and this shoot is also called a Branch referring to Jesus. The old Hebrew word netzer means sprout and is probably where the city of Nazareth got its name. The fact that Jesus was raised in Nazareth would make Him a Nazarene and so this is the fulfillment of the fourth prophecy concerning Jesus' birth and early years.