"Then those who feared the Lord talked with each other and the Lord listened and heard. A scroll of remembrance was written in his presence concerning those who feared the Lord and honored his name." NIV translation
Then, some of God's people repented and turned back to Him. God is faithful and forgave them just as He does with us. God has always preserved a remnant of His people.
"'They will be mine,' says the Lord Almighty, 'in the day when I make up my treasured possession. I will spare them, just as in compassion a man spares his son who serves him. And you again will see the distinction between the righteous and the wicked, between those who serve God and those who do not.'" NIV translation
The final answer to the question of why they should serve the Lord is given in this passage. On the day of judgment, there will be a separation of the righteous from the wicked. The righteous will live with God for all of eternity while the wicked will be doomed to hell.