Anthropic has found a way to shed new light on how its models solve problems, thanks to its discovery of what it has dubbed the J-space.
“We find that Claude has developed a small collection of internal neural patterns that, compared to all its other internal processing, play a special role. We call the collection of these patterns the J-space, named after the technique we used to find them, involving a mathematical concept called the Jacobian,” Anthropic said in its post about the discovery. It examines the contents of the J-space using what it calls the Jacobian lens, or J-lens.
“Each J-space pattern is linked to a particular word,” Anthropic said. “But when one of these patterns lights up, it doesn’t mean the model is saying that word, just that the word is on its ‘mind.’ If you’ve heard of language models having a scratchpad or chain of thought—text they write to themselves while reasoning—the J-space is something different. It operates silently, in the model’s internal neural activations, allowing the model to ‘think’ about a concept without writing it down.”
This new level of analytical visibility goes well beyond what Anthropic announced as an internal scratchpad for its models in 2024. That scratchpad revealed what the model was considering when preparing an action or delivering an answer. The new development instead focuses on something much deeper which has the potential to change how AI systems are evaluated and purchased.
