Five rescue helicopters were sent to the scene, 2,900m (9.500ft) above sea level in the Zillertal Alps.
The climbers were roped together when one of them slipped, pulling the rest of the group with him, police said.
They fell 200m (650ft) down a slope before landing in a crevasse. Only one survived, and was flown to hospital in Salzburg, about 80km (50 miles) away.
Officials say the climbers were below the Mannlkarscharte pass near Krimml, in an area where a heightened risk of rockslides made recovery of the bodies more difficult.
"We think a rope may have come loose," said Anton Voithofer, head of the Red Cross rescue team.
Three of those who died were over 60 years old. The climber who was seriously injured is in his 70s.
The names and nationalities of the victims have not been released at this stage.
The accident is believed to be the deadliest so far this season in the Austrian Alps, where several dozen mountaineers die annually.
In a separate accident, three Italian climbers are believed to have died on a glacier in the Ademello Brenta Park near Trento, in the Italian Alps.
Two of them perished after falling into a crevasse. The Associated Press reports that a third was rescued but died later.
They were climbing as a group of nine mountaineers roped together, and their six companions all sustained injuries.
On Saturday, an Italian climber died after he was hit by rocks in the Valtellina valley near the Swiss border.
Photp by: AFP/GETTY IMAGES Caption: A view of the Zillertal Alps range in Austria