This is a Plain English Papers summary of a research paper called How Cross-Model Compatibility Lets Attackers Extract Proprietary LLM Reasoning Traces. If you like these kinds of analyses, you can find more research on AIModels.fyi or follow us on Twitter.
The illusion of safety
Major AI companies now show users their models' step-by-step reasoning as a feature. OpenAI offers it through o1, Anthropic through extended thinking, Google through its reasoning-focused variants. But this reasoning is a double-edged sword. It's intellectually valuable to share, showing users why a model reached a conclusion. But it's also intellectually valuable to steal. Competitors want to understand how frontier models think. Researchers want to study their reasoning patterns. Attackers want to extract proprietary algorithms.
So the companies made a choice: hide the reasoning from users by encrypting it. The idea sounds straightforward enough. Return the reasoning to the user's device in an encrypted, unreadable form. The user can't see it, competitors can't see it, but they can pass it back to the server in future requests if they need continuity with previous reasoning. The server alone holds the decryption keys. Problem solved.
