Brigades largely composed of fighters from Misrata advanced this week to the perimeter of Sirte's city center, after launching a counter attack against ISIL (also known as Daesh) last month and driving the militant group back along the coastal road between the two cities.
The brigades are part of an operation backed by Libya's UN-backed Government of National Accord (GNA), which arrived in Tripoli in March and has been gradually working to establish its authority.
Western powers see it as the best bet for trying to unite Libya's political and armed factions against ISIL and for restoring some stability to the oil-rich North African country.
A source from the operations room in Misrata said fighters from the front line in the south of Sirte had looped round to the seafront to capture the port, which lies about 5 km (3 miles) east of the city center.
The brigades have advanced more swiftly than many expected, though their progress has been hampered by suicide bombers, mines and snipers, Reuters reported.
Clashes in Sirte on Friday left 11 brigade members dead and 35 wounded, the operations room source said. More than 100 fighters from the GNA-backed brigades have been killed and more than 500 wounded since the campaign to recapture Sirte began in early May.
ISIL started expanding into Libya in 2014 as the political turmoil and conflict in the country worsened. It took full control of Sirte last year, but has struggled to retain territory or win support elsewhere in Libya.
As the brigades based in Misrata have advanced on Sirte from the west, a separate force that controls some of Libya's key oil terminals and is also aligned with the GNA has pushed Islamic State back from the east. They reached Harawa, a town about 70 km from Sirte, on Thursday.