John Galliano has bowed out of the Met Gala where he was set to be honored, after a backlash over past remarks by the veteran designer, including an antisemitic rant.

“After much reflection and discussion with all those involved, I have decided, with great sadness, that it is best for the exhibition not to take place at this time,” Galliano said Monday in an Instagram post. The exhibit that has been set to kick off May 3 with the Met Gala is overseen by Anna Wintour, Vogue’s global editorial director who also serves as a Met trustee. But a backlash over plans to honor Galliano, who is credited with growing luxury-fashion brand Dior into a global giant, left Met Gala organizers in damage control.

“I do not want the debate surrounding me to place the Met in a difficult position or to distract from the remarkable work of the Costume Institute. I also recognize and respect that the exhibition honoring my work would be painful for some,” Galliano added in his statement.

Galliano was fired by the company in 2011 after video footage surfaced showing the designer, who was inebriated, making antisemitic remarks in a Paris café. Galliano retreated from public life for two years and was found guilty by a Paris court of public insults based on religion, race and ethnicity.