“Interpretability” names three research programmes that share a word and almost nothing else. One wants to reverse-engineer the computation into an algorithm. One wants to know what information a representation carries. One wants an explanation that satisfies somebody outside the lab. They use different methods, and — the part that matters — they would accept different things as proof.
Three programmes, not one field
Mechanistic interpretability tries to recover the algorithm. The claim it aims at has the form: this set of attention heads and MLP neurons, wired this way, computes this function, and here is the intervention that confirms it. The unit of output is a circuit. Work in this programme is dominated by small models — two-layer attention-only transformers, GPT-2 small, toy networks trained on modular arithmetic — because a claim about an algorithm has to be checked exhaustively and that is only tractable when the model is small.
Representation analysis asks a weaker and much more answerable question: what information is present in this activation, and is it used? Probing classifiers, representation-similarity measures and concept-vector methods live here. The output is not an algorithm but a map — part-of-speech information is linearly recoverable by layer four, sentiment by layer eight, and here is the direction along which it lies.
