The argument doesn't wash. Excavations in and around the Old City have revealed many other Herodian structures, including further remnants of his wall. Besides, as we read in Wikipedia:
It has been a site for Jewish prayer and pilgrimage for centuries, the earliest source mentioning Jewish attachment to the site dating from the 4th century.That would be at least 200 years before Muhammad and Islam.
Some Palestinian experts confine themselves to the more elegant and also scripturally sound argument that their forebears the Canaanites were probably worshiping in the Jebusite city now known as Jerusalem long before the Jews even got there. As for proving Jewish provenance on the Temple Mount, all we have to do is excavate beneath it, which I doubt the Muslim authorities would countenance.