Typhoon Hagibis made landfall south of Tokyo on Saturday, moving north and bringing severe flooding.
Thirteen people are missing from the storm, public broadcaster NHK said.
In central Nagano prefecture, water surrounded Japan's famous bullet trains while helicopters plucked stranded residents from rooftops.
A total of 27,000 military troops and other rescue crews have been deployed in relief operations, the government said.
In Kawagoe city, north of Tokyo, emergency crews used boats to help residents trapped in a nursing home.
About 400,000 homes lost power with running water also hit. Train and flight services cancelled under the threat of Hagibis are resuming.
By Sunday the storm had weakened and moved off land.
The storm led to some Rugby World Cup matches being cancelled but a key fixture between Japan and Scotland will go ahead on Sunday.
Qualifying for Japan's Formula One Grand Prix was also disrupted but the race went ahead.