"Then, the fifth time, Sanballat sent his aide to me with the same message, and in his hand was an unsealed letter in which was written: 'It is reported among the nations - and Geshem says it is true - that you and the Jews are plotting to revolt, and therefore you are building the wall. Moreover, according to these reports you are about to become their king and have even appointed prophets to make this proclamation about you in Jerusalem: "There is a king in Judah!" Now this report will get back to the king; so come, let us confer together.'" NIV translation
The enemy could not get Nehemiah to compromise so, on the fifth try, they used gossip and threats. Basically, they were saying that Nehemiah was going to be the king of Judah as they were going to rebel against King Artaxerxes. Sanballat basically said that he was going to pass on the lies unless he agreed to meet and compromise. The penalty for rebellion against the king would have been death so Nehemiah was really putting his life on the line here.