Mayor Rahm Emanuel announced during a news conference on Tuesday that he had asked Garry McCarthy, police superintendent since May 2011, to resign, Reuters reported.
Emanuel also said he was creating a new police accountability task force.
On November 24, the video was finally released which showed the shooting of 17-year-old Laquan McDonald by officer Jason Van Dyke on Oct. 20, 2014. The officer was charged with first-degree murder in the death of McDonald.
High-profile killings of black men at the hands of mainly white law enforcement officers in US cities have fueled demonstrations for some two years, stoking a national debate on race relations and police tactics.
Emanuel, a Democrat and former chief of staff to President Barack Obama, said he was responsible for what happened in the case, the same as the police superintendent. He said the creation of the task force was meant to rebuild trust in the police department of one of the country's largest cities.
In an editorial on Tuesday, the Chicago Sun-Times had called for McCarthy's resignation and the Chicago City Council black caucus and some protesters had also called for him to leave.
About 150 demonstrators endured nearly freezing temperatures to rally outside Chicago police headquarters on Tuesday evening, in a protest led by the city's chapter of the Black Lives Matter group.
"We have built a resistance movement and put pressure at the highest offices in both the police department and City Hall," said demonstrator Damon Williams, 23.
"This is a very long fight and we have a lot more work to do."