You've probably watched an AI think through a problem step by step, nod along with the logic, and land on an answer — and never once asked whether the steps actually did anything. What happens if you cut the reasoning off halfway through and force an answer anyway? What happens if you slip a wrong number into the middle of it and let it keep going? Does it actually get derailed, or does it already know where it's headed?
That question has a name in AI safety research: chain-of-thought faithfulness — whether a model's stated reasoning is the real computation behind its answer, or a plausible-sounding narrative generated after the fact, with the conclusion decided before the "steps" ever got typed out.
Anthropic ran a formal version of this in 2023: Measuring Faithfulness in Chain-of-Thought Reasoning (Lanham et al., arXiv:2307.13702). Two of their four tests:
Early Answering — truncate the reasoning partway, force a final answer, see if it matches the full-reasoning answer.
Adding Mistakes — inject a wrong step into the reasoning, let the model continue from there, see if the error survives to the final answer.
