He met his Iraqi counterpart Ibrahim al-Jaafari, a foreign ministry official told AFP on condition of anonymity.
Another government official said it was the first visit to Baghdad by a Saudi foreign minister since 2003, marking a key step in efforts to mend frosty bilateral relations.
"This visit is to reestablish relations in a more stable way than previously," a senior government official said. "It's the first visit of its kind."
Iraq Prime Minister Haider al-Abadi, who has been at the helm since 2014, has supported efforts to improve strained ties.
Thamer al-Sabhan, whose credentials were received in January 2016, became the first Saudi ambassador to Iraq in a quarter century, after relations were cut following ex-president Saddam Hussein's invasion of Kuwait.
But he left the same year after Baghdad demanded he be removed following remarks he made to the press about an alleged plot to assassinate him and criticism he voiced of the Hashed al-Shaabi (Popular Mobilization Forces).