Zidane was one of the favourites to take charge of the national team if Didier Deschamps had stepped down but his contract was extended after he led France to the World Cup final, where they lost to Argentina in Qatar last month.
When asked if Zidane, a World Cup winner with France in 1998 and a national icon, would now manage the Brazilian national team instead, Le Graet had told RMC: "I don't give a damn, he can go wherever he wants."
Le Graet added he would not even pick up the phone if Zidane called, prompting a backlash and his subsequent apology.
"I would like to present my personal apologies for these remarks which absolutely do not reflect my thoughts, nor my consideration for the player he was and the coach he has become," Le Graet said in a statement reported by L'Equipe.
"I admit that I made awkward remarks which created a misunderstanding. Zidane knows the immense esteem I have for him, like all French people."
On Sunday, Le Graet had said Zidane was on the radar and had plenty of support but stood no chance of replacing Deschamps.
"Who can make serious reproaches to Deschamps? Nobody... If Zidane tried to contact me? Certainly not, I wouldn't even pick up the phone," he had said.
Zidane was the first coach to win the Champions League three times in a row when he was in charge of Real Madrid.
The 50-year-old, who also won the 2000 European Championship with France in a glittering playing career, is currently without a job after he left his role with the Spanish club last year.
'LACK OF RESPECT'
France forward Kylian Mbappe was one of the first to voice his disapproval after Le Graet's comments were aired.
"Zidane is France, we don't disrespect the legend like that," Mbappe said on Twitter.
Le Graet's dismissal of Zidane as a potential successor to Deschamps comes two years after the 81-year-old official had backed the former midfielder to take up the reins if he was available.
Apart from coaching Real, Zidane also spent five seasons as a player at the LaLiga club where he won the league and Champions League. The Spanish club described Le Graet's comments as 'unfortunate'.
"These remarks show a lack of respect for one of the most admired figures by football fans around the world and our club is awaiting an immediate correction," Real Madrid said.
"The statements by the president of the French Football Federation are inappropriate for someone holding such representation and are in themselves unsuitable."
France's Minister for Sports Amelie Oudea-Castera was one of many politicians who reacted to Le Graet's comments, saying the president of the country's "biggest sporting federation" had crossed a line.
"Yet more out-of-touch comments and on top of that a shameful lack of respect, which hurts us all, towards a legend of football and sport," she wrote on Twitter.
French politician Pieyre-Alexandre Anglade had also demanded an apology from Le Graet, saying on Twitter: "Zidane is a monument of French football and sport. A personality loved by all French people... Our football deserves better than that."