Mr. O'Neill held a press conference on Sunday in Port Moresby, indicating that he would resign "in the coming days".
After almost eight years in the position, he said he would hand over the leadership to Sir Julius Chan, a 79-year-old who has had two previous stints as prime minister.
The prime minister's resignation is not final until after it is received in writing by the governor general. Mr O'Neill said he would visit the Queen's representative this week, to "clear the way for the parliament to vote for the next prime minister".
However, the prime minister conceded that recent political movements had indicated to him there was a need for change in leadership.
Pressure has been building for weeks on Mr O'Neill's coalition government with an exodus of MPs, including senior ministers, from his People's National Congress party, joining the opposition.
As of Friday, with the defection of William Duma's United Resources Party, the opposition was claiming to have 62 MPs in the 111-seat parliament, as it sought to oust the prime minister by a parliamentary motion.
Mr O'Neill appeared alongside Sir Julius and other leaders of coalition parties on Sunday.
"We have agreed to a change of direction, that the leadership of our government will be now handed over to Sir Julius Chan, who is a veteran leader and one of the founding fathers of our great nation," Mr O'Neill said at the press conference.
"In consultation with coalition government partners, we have decided to ask Sir Julius Chan to lead the team in government for the remainder of this term of parliament," Mr O'Neill said in a statement issued later on Sunday.
Usually, under provisions of PNG's constitution, the deputy prime minister takes up the vacancy when a prime minister steps down. In this case, Deputy Prime Minister Charles Abel has been overlooked by Mr O'Neill in favour of the leader of a coalition partner, the People's Progress Party.
The plan to pass the reins to Sir Julius comes after Mr O'Neill recently lost the large majority support he'd enjoyed in parliament since 2011, as a flood of grievances over PNG's ailing economy, deteriorating basic services and festering corruption allegations finally turned the tide against him.
'Government in waiting'
Following Mr O'Neill's announcement, the opposition held a press conference at its Laguna Hotel base. Leading figures in the group said they would not believe Mr O'Neill's announcement until he formally resigned.
Opposition power broker James Marape, whose resignation as Finance Minister last month sparked the exodus, cautioned over "mixed signals" from the government.
"There is no such thing as the prime minister resigning and handing over leadership to someone who is not even a minister of state. That is legally not correct."
Leading opposition MPs described their group as a government in waiting. Over recent weeks, lobbying between MPs has been intense, with at least two more government MPs joining the opposition today.
Environment Minister John Pundari made it to the Laguna just before the opposition decided to lock the gates of the complex at midday, while another pair of government MPs looking to join the fray were turned away.
But Mr O'Neill, speaking from his base at the Crown Hotel, argued that maintaining a government based around his People's National Congress and the remnants of his coalition would be best for the interests of political stability.
"There is no way that I could stand by and allow the opposition to come into government with their dangerous mix of wild ideas," Mr O'Neill said.
A political, and potentially constitutional, crisis is brewing, because Mr. O'Neill's move to hand over the role of prime minister to Sir Julius will not be readily accepted by opposition MPs.
Mr Marape warned that attempts could be made by the O'Neill regime to sabotage processes of parliament at this important juncture. Yet with the opposition appearing to have a majority, a vote for a new prime minister is likely in the coming days once parliament resumes on Tuesday.