He made the decision after seeing that "certain political factions are not yet ready reform" and that continuing his tenure under such circumstances would run contrary to his promises to the Iraqi people, Allawi said in a speech on Sunday.
He asked the Iraqi presidency to excuse him from forming a new government, according to Press TV.
Allawi's withdrawal comes a few hours after Iraq’s parliament failed to reach a quorum necessary to hold a vote-of-confidence session for his new government.
Following the announcement, Iraqi media reported that President Barham Salih will begin consultations to choose a new candidate for prime minister .
Allawi was tasked early February with forming a new government after former Prime Minister Adel Abdul Mahdi resigned in the wake of unprecedented anti-government protests which accused the former government of corruption and mismanagement.
The demonstrations, which started in early October last year, have since regularly turned violent amid reports of foreign intervention.
Several hundred people, including security forces, have lost their lives in the violence.