The bombing on Thursday was claimed by the Taliban, which called it a "revenge attack" for a US airstrike in the province of Kunduz earlier this month that left up to 32 civilians dead.
US forces admitted last week that its airstrikes "very likely" resulted in civilian killings in Kunduz, pledging a full investigation into the incident.
According to Mazar-i-Sharif police, "the suicide attacker rammed his explosives-laden car into the wall of German consulate in the city".
"The blast was too loud and powerful, which shattered windows, and many civilians were wounded inside their homes," Abdul Raziq Qaderi, head of security for Balkh province where Mazar-i-Sharif is located, said.
A German military spokesperson said that shots were also heard outside the consulate and that NATO troops were on the site.
The number of attackers remains unclear.
Afghan police said that they found a second attacker hidden or buried under rubble on Friday morning, having spoken of only one attacker during the night.
A Taliban spokesman said that several fighters had entered the building and that clashes were ongoing, Al Jazeera reported.
The government source, however, did not confirm if fighting was still ongoing.
Germany has 938 soldiers stationed in Afghanistan, most of them in Balkh, as part of NATO's Resolute Support mission.
The Taliban, who seized power and ruled Afghanistan from 1996, were toppled by a US-led invasion after the September 11, 2001, attacks on New York and Washington.
Almost 15 years later, about 13,000 troops from a US-NATO coalition remain in the country.
In the past year alone, the Taliban's armed campaign has killed or wounded about 11,000 civilians, as well as 5,500 government troops and police officers.