A dominant Brumbies scrum shoved the Waratahs pack over the line for a penalty try in the 68th minute to set up the win before replacement back Nigel Ah Wong added a last-minute try to blow out the scoreline at Canberra Stadium.
The Brumbies, widely tipped to win the Australian conference in the expanded 18-team competition, snapped a four-match losing streak against the Waratahs and remained undefeated after two rounds.
Christian Lealiifano and Joe Tomane scored the other tries for the Brumbies, with Israel Folau and Nick Phipps crossing for the visitors.
"We finished well there at the end and very happy to get the win against a quality team," Brumbies captain Stephen Moore said pitch-side.
The Waratahs were denied a try to Kurtley Beale with six minutes left due to obstruction by replacement lock Sam Lousi but it was a deserving win for the Brumbies who defended stoutly and were more fierce at the breakdown.
The Waratahs made a calamitous start, losing Will Skelton and fellow lock Dean Mumm to yellow cards in the first half.
Flyhalf Lealiifano capitalised during Skelton's time in the sin-bin for a high tackle, shimmying through an undermanned defence in the ninth minute.
A piece of Folau brilliance put the Waratahs back in the game as the Wallabies fullback sliced through to cross at the right corner in the 22nd minute.
Although a man down for a second time when Mumm went off near the hour-mark for collapsing a maul, the Waratahs scored a penalty against the run of play to claw back to 8-8 at the break.
The Brumbies raised the pressure and Tomane shrugged off two tacklers in a dart to the line in the 56th minute.
Waratahs scrumhalf Nick Phipps hit back moments later, capitalising on a Folau line-break for his team's third try.
The Brumbies forwards then took control of the match, before Tomane won a sprint to collect a loose ball and threw a hopeful pass over his shoulder which Ah Wong gleefully swooped on before bolting 30 metres for the final try.