Two Black men who were fired by President Donald Trump from the National Transportation Safety Board and U.S. Surface Transportation Board accused the administration on Thursday of discriminating against them as part of a pattern of dismissing Black leaders across the government.

Robert Primus on the STB and Alvin Brown on the NTSB were the only Black board members overseeing their officially independent agencies when they were fired this year, in August and in May. Both had already filed lawsuits challenging their dismissals, saying the White House didn’t have good cause, as the law requires. Democracy Forward filed the new discrimination claims on behalf both men.

“When you look at who has been removed without cause, and who has been left in place, the pattern is impossible to ignore: Black commissioners across the federal government have been summarily fired,” said Brown, who was Vice Chairman of the NTSB. “My abrupt removal was unlawful, and it was discriminatory.

The White House didn’t immediately respond to the new legal filing, but has said Trump was well within his legal rights to fire Primus and Brown. The administration hasn’t filed a formal response to Primus’ lawsuit yet, but the Trump administration asked a judge to dismiss Brown’s lawsuit, arguing that the statutory protection saying board members can only be fired for cause is unconstitutional, and that the president should be able to pick his team at every executive agency.