Sixth seeds Venus and German partner Tim Puetz were beaten by Australian wildcards Nick Kyrgios and Thanasi Kokkinakis 7-5, 3-6, 6-3 in the men's doubles, before he and compatriot Erin Routliffe were beaten in the mixed doubles.
The Kiwi pair went down 6-4, 6-2 to fifth seeds Kristina Mladenovic of France and Ivan Dodig of Croatia.
However it was the men's doubles clash that attracted another raucous crowd on Kia Arena with the locals winning in 2 hours and 16 minutes.
On court afterwards Kyrgios paid credit to Venus and Puetz saying 'they were the toughest pair they have played at the tournament.'
Both pairs had their chances in the first set.
Venus saved a break point in the opening service game with an audacious second serve at 30-40 and then Kokkinakis saved a couple of break points at 3-4.
At 5-5 Puetz found himself in big trouble, a couple of double faults had him 15-40 and while he saved the first break point, Venus netted a forehand volley and Kokkinakis after getting the break served out the opening set in 51 minutes.
Kyrgios had served superbly and was untroubled throughout the first set. The Australian pair sent down 13 aces and only dropped two points on first serve.
But after Venus held serve in the opening game of the second set, Kyrgios with a couple of loose points found himself 30-40. A wonderful return of serve from Puetz set up Venus to put away the forehand volley and get an early break.
They raced to a 5-2 lead and had three set points on the Kokkinakis serve but the Aussie held leaving Venus to try to close out the set.
From 0-40 he dug himself out of trouble and held to send the match to a decider.
Games went with serve until the 4th game when Puetz was broken to love and after Kyrgios held to love, the Australian pair led 4-1. After Venus held serve Kokkinakis faced a break point in the next game but held for a 5-2 lead. Puetz held comfortably and it was left to Kyrgios to close out the match which he did with some thunderous serving.
The Australians converted 2 of 11 break point chances in the match with Venus and Puetz breaking just once from six opportunities.
Kyrgios said he was surprised that the host broadcaster opted to show their doubles match instead of Rafa Nadal's singles quarter-final.
When the local favourites stepped on to court, Nadal was leading Canadian Denis Shapovalov by two sets at the Rod Laver Arena main showcourt, with the third set tied at 3-all.
Channel 9, which has the broadcast rights to the Australian Open, decided to switch to the "Special Ks", the nickname given to the pair when they were promising teenagers.
"Nadal is a hell of a player. That's unexpected," Kyrgios said. "Shapovalov and Nadal are two probably of the best players in the world right now.
"But, I mean, I think the level of entertainment ... is different, I think."
After their second round win, Kyrgios said tennis administrators should cultivate players who were capable of extending the appeal of the sport beyond traditional audiences.
Nadal, a 20-times major champion, defeated Shapovalov in a five set thriller 6-3 6-4 4-6 3-6 6-3.
Photo PHOTOSPORT Caption: Micheal Venus (left)