Thirteen people were also missing, media reported on Wednesday, after a month's worth of rain fell overnight, loosening slopes already saturated by heavy rain over the past few weeks, Reuters reported.
"There was rain and thunder all night, beating down so hard I was scared to go outside," a resident told Fuji TV. "I've never seen anything like this."
Helicopters clattered overhead, lifting out survivors, as rescue workers searched through mud and piles of stones in residential areas about 5km from the city centre.
Those dug out of the debris included a two-year-old boy and his 11-year-old brother, whose house was struck as they slept.
With land in short supply in many parts of Japan, cities often expand into mountainous areas, leaving such development vulnerable to landslides.
About 240mm of rain fell in the area in the 24 hours up to Wednesday morning, record-breaking levels equivalent to a month's worth of rain in a usual August, the Meteorological Agency said.
Asphalt roads crumbled under the force of one landslide, while streams of mud cut swathes through neighbourhoods, turning houses into piles of twisted wreckage.
Prime Minister Shinzo Abe cut short his summer vacation to head back to Tokyo and said he would dispatch several hundred military personnel to help with rescue efforts.
Landslides killed 31 people in Hiroshima in 1999.