"The fact that, for the first time in the history of relations between the United States and France, we are recalling our ambassador for consultations is a grave political act that shows the intensity of the crisis today between our two countries and also with Australia," Le Drian said.
A White House official has said the Biden administration will engage with France in the coming days to resolve their differences.
Australian Foreign Minister Marise Payne said she understood the "disappointment" in France and hoped to work with the country to ensure it understood "the value we place on the bilateral relationship".
Earlier, France recalled its ambassadors to the United States and Australia for consultations, in a backlash over a submarine deal.
Australia on Wednesday cancelled a multibillion-dollar deal for conventional French submarines to enter a strategic Indo-Pacific alliance with the United States and Britain in which it will obtain US nuclear-powered submarines.