Coastguard spokesman Ayoub Qassem said 23 migrants were rescued from the craft off Gargaresh, a western suburb of Tripoli. Survivors said the inflatable boat had set off with about 120 people on board.
"Some 97 are still missing, including 15 women and children," Qassem said, Reuters reported. "What happened is that the base of the boat got wrecked and the boat had sunk."
Libya is the main departure point for migrants hoping to reach Europe by sea, and more than 150,000 have made the crossing from Libya to Italy in each of the past three years.
Smugglers put most of the migrants in flimsy inflatable dinghies that get picked up by rescue ships and other vessels once they reach international waters. Some are intercepted and turned back by the Libyan coastguard.