"The whole company numbered 42,360, besides their 7,337 menservants and maidservants; and they also had 245 men and women singers. There were 736 horses, 245 mules, 435 camels and 6,720 donkeys." NIV translation
Here, we are given the number of people that returned to the land and we notice that it was significantly less than the total number of Jews.
"When they arrived at the house of the Lord in Jerusalem, some of the heads of the families gave freewill offerings toward the rebuilding of the house of God on its site. According to their ability they gave to the treasury for this work 61,000 drachmas of gold, 5,000 minas of silver and 1oo priestly garments." NIV translation
We see that God spoke to the hearts of some of the families and used them to provide the money to rebuild the temple. We see that this was a "freewill" offering which means that it was done spontaneously and willingly as prompted by the Holy Spirit. Today, we have corrupted the concept of a freewill offering as many leaders badger people into giving instead of allowing God to speak to the hearts of men to provide for His work.
As the saying goes: 'If it's His will, it's His bill" and so we can trust God to provide what is required for the building of His kingdom.
"The priests, the Levites, the singers, the gatekeepers and the temple servants settled in their own towns, along with some of the other people, and the rest of the Israelites settled in their towns." NIV translation
So, the people returned to their towns.