We ran 160 agent tasks across two frameworks. The frameworks tied. Then we changed the model.

We ran 160 agent tasks across two frameworks. The frameworks tied. Then we changed the model — and one task failed 20 times in a row.

The LangChain vs LangGraph question comes up every time someone starts a new agent project. We had a slightly different version of it: once you've chosen LangGraph as your runtime, does picking the library around it actually change production outcomes?

So we ran a controlled benchmark to find out.

The setup

LangGraph 1.2.9 against Pydantic AI 2.13.0. Four tasks, designed to test realistic agent work: order processing with tool calls, shipping quote calculation, refund eligibility with date logic, and inventory reorder decisions. All tasks shared the same tool implementations and the same scorer — the only variable was which library orchestrated the calls.