Typhoon Haishen is expected to intensify on Sunday, bringing heavy rain, storm surges and winds of more than 160kmh.
It will move past Kyushu on Sunday, and is expected make landfall in South Korea on Monday.
It comes days after Typhoon Maysak, one of the strongest storms to hit the region in years.
The latest typhoon has led to the closure of factories, schools and businesses across western Japan. Hundreds of flights and train services have also been cancelled.
Japan's government will be holding an emergency cabinet meeting on Sunday to address the storm.
The country's weather agency said it was not likely to issue its most serious typhoon warning, but added that residents should exercise their "most serious caution" for possible record rains and high waves along the coast.
Yoshihisa Nakamoto, director of the agency's forecast division, told reporters that surging tides may also cause flooding in low-lying areas, particularly around river mouths.
As a result of the storm, hundreds of thousands of people have been advised to move to designated shelters in places like schools and community centres. This includes all 36,600 residents of the city of Goto, in Nagasaki, where the typhoon may hit directly.
But local media report that some people have chosen to find safety in local hotels to reduce the risk of coronavirus spreading in crowded public shelters.
Typhoon Haishen has also forced Japan's coast guard to suspend its search for missing sailors from a cargo ship that sank during Typhoon Maysak.
The Gulf Livestock 1 was carrying 43 crew members and 6000 cows when it went missing on Wednesday.