The militants targeted the Temple of Bel, a Roman-era structure in the central desert city, the so-called Syrian Observatory for Human Rights said Sunday.
It is the second temple ISIL has targeted in Palmyra this month. The group detonated explosives in the ancient Baal Shamin temple on Aug. 25, an act that cultural agency UNESCO has called a war crime aimed at wiping out a symbol of Syria's diverse cultural heritage, Reuters reported on Monday.
The extent of the damage at the Temple of Bel was not known, the Observatory said, citing its contacts on the ground.
Activists on social media also reported the destruction at the temple, one of Palmyra's most important structures.
ISIL seized Palmyra in May from Syrian army in a sudden offensive and is tightly controlling communications in the city, according to activists.
The group, which has proclaimed a caliphate in territory it holds across Syria and Iraq, has regularly demolished monuments it considers sacrilegious as well as carrying out mass killings.
This month the group beheaded the 82-year-old Syrian archaeologist who had looked after Palmyra's ruins for four decades, and hung his body in public, according to Syria's antiquities chief.
A general view of the Temple of Bel in the historical city of Palmyra April 18, 2008.