At least two bull sharks were spotted in the Shelly Beach area, in northern Sydney, after the attack and authorities closed all nearby beaches as a precaution.
BBC reports the injured dolphin circled the shallow waters but eventually beached and died.
Manly Open Surf, a carnival taking place at the beach over the weekend, has been suspended.
Lifeguards cleared people from the water after the attack, which happened at around 07:00 local time on Saturday (20:00 GMT on Friday).
Hundreds of people had been about to compete in the festival.
Tracey Hare-Boyd, a spokeswoman from Surf Life Saving New South Wales (SLSNSW), said that, luckily, no person had been injured or involved in the attack.
SLSNSW said it was monitoring the shark activity via a drone and urged people not to enter the water until it was deemed safe by lifeguards.
An eyewitness who was on Shelly beach told ABC News she was initially delighted to see a dolphin swimming in the bay.
But she soon learned there was a shark in the water and swimmers were told to stay on dry land.
"We came back and saw the dolphin was on the beach which was really incredibly sad," she said.
It was not yet clear whether the dolphin died from injuries sustained in the attack or whether it was already ill and dying, which would have made it an easy target for the sharks, SLSNSW's Ms Hare-Boyd said.
Beaches from Shelly Beach to Queenscliff Beach remain closed and people have been urged to be vigilant and follow shark safety advice.
Human deaths from shark attacks are rare in Sydney, with only two in the past 60 years.
Last February, British swimmer Simon Nellist died in the city's first fatal shark attack in 59 years after he was mauled by a great white in the waters off east Sydney.