It wasn’t immediately clear how far the assailant got inside the building, which is located across the street from Notre Dame Cathedral, AP reported.
Police union official Loic Travers said one police officer suffered fatal wounds and another officer shot the attacker “in a situation that appears to be self-defense.”
The motives of the midday attack were not known.
Another police union official, Jean-Marc Bailleul, told French broadcaster BFM TV that information was still coming in but he understood the attacker was a police department employee.
French media reported that a staff member carried out the attack with a ceramic knife in part of the prefecture that is not open to the public.
The attack came a day after thousands of officers marched in Paris to protest low wages, long hours and increasing suicides in their ranks.
France’s interior minister and the Paris prosecutor were on their way to the scene.
Extremists have repeatedly targeted French police in France in recent years. In 2017, a gunman opened fire on the Champs-Elysees Boulevard, killing one officer before he was shot to death.
In 2016, an attack inspired by the Daesh (also known as ISIL or ISIS) group killed a police officer and his companion, an administrator, at their home in front of their child.