Ireland led 10-7 at half time, with the All Blacks having a player sent off after 30 minutes.
Prop Andrew Porter scored the opening try for Ireland after just three minutes, with Jonny Sexton kicking the conversion and then a penalty to give the visitors a deserved early lead.
The All Blacks then had winger Leicester Fainga'anuku and Ofa Tu'ungafasi both sin-binned and replacement prop Angus Ta'avao was red-carded in the 31st minute for a dangerous tackle.
All Blacks number eight Ardie Savea was forced off the field when Tu'ungafasi returned but the Irish were unable to take advantage and ended the first half on the back foot.
The home side was able to finish the first spell strongly, turning strong defence into attack in the last five minutes with Beauden Barrett getting them back into the game with a try under the posts to end the half.
Ireland lock James Ryan was shown a yellow card during the All Blacks siege, leaving both sides with 14 men for the first 10 minutes after the break.
The visitors were 17-7 ahead by the time he returned after another multi-phase wave of attacks resulted in Porter barging his way over the line for his second try.
Ireland kept their composure as the All Blacks came back at them and Sexton kept the scoreboard ticking over with two more penalties, the second 12 minutes from time to take the lead out to 16 points.
New Zealand kept battering away at the Irish defence in the final 10 minutes and finally managed to get the ball to Will Jordan in space with two minutes on the clock.
It is the fourth time Ireland have beaten the All Blacks since 2016, but the first time they've won in New Zealand. Ireland have now won four of the last seven encounters with the three-times world champions.
The All Blacks won the first test 42-19, so the series will be decided in Wellington next weekend.
"Delighted, no team has ever done it before," said Sexton. "We're delighted with the win but we have a chance to win a series here and they don't come along too often."
"They're a quality outfit and they bounced back in a big way tonight," said All Blacks captain Sam Cane.
"They were too good tonight."