The fire department was alerted to the blaze in the three-storey building by an emergency call at around 1:30 am (1630 GMT Saturday). Four bodies were found after it was extinguished.
Four other men, aged between their 40s and 70s, “were taken to hospital unconscious”, Masatoshi Sumitani at the Kobe fire department said, AFP reported.
Most of the occupants of the building were believed to be elderly men, Japanese media reports said, citing other residents of the building.
Black smoke could be seen coming out of a window on the first floor when the fire broke out, Kyodo News reported.
Police will begin investigating the cause of the fire later on Sunday, Sumitani said.