OpenAI says that two Responses API settings, retained reasoning and compaction, raised GPT-5.6 Sol's public-set ARC-AGI-3 score from 13.3% to 38.3%. The result matters because it shifts part of the explanation for a weak benchmark outcome away from the model alone and toward the software harness used to run it. OpenAI also reports that the configuration reduced output tokens by about six times.
The company had previously presented GPT-5.6 Sol as capable of advanced mathematical work, including attempts at open mathematical problems. Its ARC-AGI-3 investigation instead found that the evaluation setup was not allowing the model to retain what it had learned while working through the benchmark's 2D puzzle games. In OpenAI's detailed ARC-AGI-3 write-up, the company argues that API configuration can materially affect both agent performance and token efficiency.
This is an important deployment lesson, not simply a higher benchmark number. For developers building agents that must reason over multiple steps, the surrounding system determines whether useful intermediate work persists, how context is managed, and how much output is consumed to reach a result.
Why the harness changed the result
ARC-AGI-3 is designed around interactive 2D puzzle games, a setting that can require an agent to try actions, observe outcomes, and adapt its approach. OpenAI's analysis says the initial harness created a memory-related limitation: GPT-5.6 Sol could not effectively carry forward relevant reasoning from earlier work.
