If you’ve ever tried chatbots in multiple languages, you already know the languages have slightly different personalities. As part of a new report on behavior inconsistencies published on Monday, Anthropic researchers acknowledged this quirk. Rather unsettlingly, they note that due to differences in the attributes of texts the models are trained on, the differences might run deeper than just tone, and might actually change the model’s priorities. These “imbalances in quantity and composition could lead Claude to express different values in different languages,” Anthropic’s researchers write. But if you’re looking for any specific examples of the models showing, say, inconsistent moral reasoning across languages, nothing of the sort is in this paper. That might involve scrutinizing direct quotes from potentially unsuspecting people.
Instead, Anthropic analyzed 309,815 chatbot conversations with the Sonnet 4.6, Opus 4.6, and Opus 4.7 models. These involved “subjective” tasks, meaning less “What’s the capital of France?” and more “How can I tell if my cat hates me?” These were anonymized, in theory, using Anthropic’s “privacy-preserving analysis tool,” and then processed (in part using Claude itself) to rate responses on a “values axis.”
