TL;DR
GradCuit (arXiv:2608.02585) inserts optimizable latent vectors at an intermediate Transformer layer and uses causal self-attention as a differentiable "circuit" to flow reward-weighted gradients directly to those latents at test time — no parameter updates, no token resampling, just smarter internal reasoning. Result: 64.5% average accuracy across 5 models and 3 benchmarks, beating Chain-of-Thought by 6.6 pp and the previous best latent-space method (LatentSeek) by 2.4 pp.
The Problem
Test-time scaling has become one of the hottest topics in LLM research. The idea is simple: spend more compute at inference to get better outputs. Chain-of-Thought, Best-of-N sampling, and self-consistency are classic examples. More recently, researchers have explored optimizing in latent space — directly adjusting the model's hidden representations without changing its weights.
LatentSeek (2505.13308) was a promising step: it uses policy gradients to iteratively update latent representations guided by self-generated rewards. But there's a fundamental flaw shared by all existing latent reasoning methods.
