Wingers Tom Wright and Jordan Petaia crossed for tries in the first half, while forwards Taniela Tupou, Rob Leota and Connal McInerney added scores after the break to maintain Australia's 100% record in six tests against Japan.
Japan, who scored tries through Lomano Lemeki and Ryoto Nakamura, drew within four points of the lead at 27-23 with five minutes remaining but debutant hooker McInerney touched down off the back of rolling maul to snuff out home hopes of an upset.
"It was a really competitive game," said Australia captain Michael Hooper. "They tested us in a lot of areas and it was a really close game in the end," he said. "They've just been improving every year."
Although far from a perfect performance, the Wallabies will be pleased to have extended their winning streak after back-to-back victories over both world champions South Africa and Argentina to conclude the Rugby Championship.
First-five Quade Cooper had a mixed match, sending Wright through to open the scoring in the seventh minute with a neat inside pass but also lofting up the pass that allowed Japan centre Ryoto Nakamura an intercept try in the second half.
The visiting pack did well to largely defuse Japan's rolling maul and a bulldozing run from Tupou finished off an innovative lineout move for a try just after the break.
Japan were playing only their third match since reaching the quarter-finals of the last World Cup and had some bright moments to delight the crowd, which showed its appreciation by rattling hand-held fans.
Number eight Kazuki Himeno said he felt emotional playing in front of home fans and added the team could have done more.
"To be honest, I'm disappointed," Himeno said. "It feels like we lost a winnable match."
Nakamura added: "Big games like this can be decided by little things. We need to keep improving and come out stronger each game."
Lemeki's try came from a delightful crosskick by Rikiya Matsuda to the corner, and there were flashes of the enterprising rugby that distinguished their 2019 World Cup campaign.
Matsuda's second penalty helped the Brave Blossoms draw within a point of the Australians at 14-13 after 33 minutes but they were always chasing the game.
After a 63-30 drubbing the last time the two teams met in 2017, however, the fact that Yu Tamura's late long-range penalty put Japan in a position to snatch a win illustrates the progress they have made in the last few years.
Both sides now move on to Europe, where Australia will face Scotland, England and Wales, and Japan will play Ireland, Portugal and the Scots in November tests.