"You have not come to a mountain that can be touched and that is burning with fire; to darkness, gloom and storm; to a trumpet blast or to such a voice speaking words that those who heard it begged that no further word be spoken to them, because they could not bear what was commanded: 'If even an animal touches the mountain, it must be stoned to death.' The sight was so terrifying that Moses said, 'I am trembling with fear.'" NIV translation
Paul gives a brief description of the events at Mt. Sinai when the Law was given to their Jewish ancestors (Exodus 19). This mountain was in the wilderness and is located in the northwest part of the modern country of Saudi Arabia. We see that this was a mountain of fear and, in fact, three thousand people were killed at this time.
"But you have come to Mount Zion, to the city of the living God, the heavenly Jerusalem. You have come to thousands upon thousands of angels in joyful assembly, to the church of the firstborn, whose names are written in heaven. You have come to God, the Judge of all, to the spirits of the righteous made perfect, to Jesus the mediator of a new covenant, and to the sprinkled blood that speaks a better word than the blood of Abel." NIV translation
The physical Mount Zion was the highest point in ancient Jerusalem. It is located south of the Old City's Armenian Quarter but was a picture of the heavenly city of Jerusalem.
This is the New Jerusalem that John speaks of in Revelation 21. In this New Jerusalem, angels are constantly praising God. It is a place where God dwells with men and the way was opened by the blood of Jesus Christ. The blood of Christ is superior to the blood of the animal that Abel sacrificed. On the Day of Pentecost, a day that had before marked the giving of the Law, three thousand people received eternal life through the power of the Holy Spirit (Acts 2).
We see a huge contrast in the fact that the law (Mt. Sinai) brought fear and death while joy and eternal life came through Mount Zion.
Even today, those who struggle with legalism are walking around in fear that they will mess up and lose their salvation. Those that keep their eyes focused on the finished work of Christ can walk in joy.