The South African Rugby Union said Small suffered a suspected heart attack late on Tuesday in Johannesburg.
Small played 47 Tests for South Africa between 1992 and 1997, scoring 27 tries, and was a key part of the team which won the World Cup on home soil in 1995.
The 15-12 victory over New Zealand in the final by South Africa's integrated team was a symbolic moment for the country, coming shortly after the end of apartheid.
In the final, Small lined up opposite New Zealand's star player Jonah Lomu and successfully prevented him from scoring.
The All Blacks had been struck down by sickness in the days before the final, with rumours of food poisoning.