The bloodbath would be the latest drastic cut to a federal agency under US President Donald Trump's administration.
When asked about US press reports of the plans to halve the IRS headcount, the source confirmed such a project was being discussed internally.
Last month, the IRS laid off around 7,000 probationary employees, amid similar cuts at other federal agencies.
Since his return to the White House in January, Trump has launched a vast offensive aimed at slashing public spending and reducing the federal bureaucracy, one of the goals he promised on the campaign trail.
To that end, he has tapped billionaire Elon Musk, his top campaign donor turned close adviser, to lead the Department of Government Efficiency (DOGE), which has cut thousands of government jobs and upended federal agencies.
Among his first targets were members of the federal bureaucracy overseeing policies that promote diversity.
Trump's return also saw the dismantling of the US Agency on International Development (USAID), ending humanitarian and international development projects around the globe.