I've been working on applying Monte Carlo Tree Search to LLM reasoning. The idea: multi-step reasoning is a sequential decision problem, and MCTS is good at those.
The Problem with Single-Shot Reasoning
When you ask an LLM a hard question, it generates one response. If that response goes down a wrong path early, there's no recovery. The model commits to its initial approach and follows it to completion, even when better alternatives existed.
This is a waste. The model might have gotten it right if it had taken a different first step. MCTS addresses this by building a tree of reasoning paths and using the UCB1 bandit algorithm to balance exploration of new paths with exploitation of promising ones.
How It Works
