The Institute of the Estonian Language (EKI) just put 60 generative AI models through a propaganda stress test. Mistral, Europe’s most prominent open-source AI company, essentially flunked.
All four Mistral models evaluated landed in the bottom third of the rankings, with the company’s best-performing model placing 47th out of 60. None of them cracked 40% efficacy in identifying or rejecting manipulative content tied to Russian propaganda narratives.
Mistral is actively negotiating a funding round reportedly worth €3 billion at a €20 billion valuation. The company positions itself as the cornerstone of European sovereign AI. And its models just demonstrated a troubling inability to resist state-sponsored disinformation.
What the benchmark actually measured
The EKI Propaganda Resistance benchmark tested models against 14 distinct Russian propaganda narratives using 75 questions delivered in three languages: English, Russian, and Estonian. Responses were scored on a 1-to-5 scale by a calibrated Claude model from Anthropic, which itself ranked among the top performers in the same test.
