Submarine Kuroshio on Thursday joined three Japanese warships in waters just southwest of the China-controlled Scarborough Shoal, the Asahi Shimbun said, AFP reported.
China claims most of the resource-rich South China Sea, through which US$5 trillion, (RM20 trillion) in shipping trade passes annually, despite competing claims from Brunei, Malaysia, the Philippines, Taiwan and Vietnam.
Tensions have been high over the Scarborough Shoal since it was seized from Manila by Beijing in 2012.
The newspaper said the submarine exercises were Tokyo's first in the South China Sea.
Japan's Maritime Self-Defense Force carried out a "practical" anti-submarine drill, including an exercise to spot enemy submarines with sonar devices, Asahi said, quoting unnamed Japanese government sources.
The sources said it was a legitimate naval exercise in neutral waters, with rights of access secured under international law.
Following the drill, the Japanese submarine plans to make a port call on Monday at Cam Ranh, central Vietnam, in a bid to display Tokyo's defense cooperation with Hanoi, Asahi said.
It will be the first call by a submarine at the strategically important port since the Second World War, it added.
The disputed South China Sea contains vital global shipping routes and what are believed to be significant oil and natural gas deposits.
China has engaged in years of land-reclamation efforts on reefs it controls in the region and built both civilian and military facilities in the contested area.
Earlier this month, Beijing lashed out at Britain for sending a warship close to the disputed islands — one of a series of "freedom of navigation" operations carried out in recent times by the US and its allies as a signal to Beijing of their right to access the disputed waters.
Immediate confirmation of the Asahi Shimbun report was not available.