TL;DR: Swapping grep for MCP retrieval tools cost my coding agent 4.1x more tokens on a 33-file repo, and saved 86% on a 249-file one. Same model, same tasks, opposite sign.
I spent a weekend replicating the agent experiment from a paper called CodeNib, on a Windows laptop with no GPU, against two repositories I wrote myself. Eight behavioral questions, four arms, 32 agent runs, every token read straight off the provider's response.
Two rules before I wrote any code. Measure, never estimate. And a result that contradicts the paper is the interesting result, so do not massage it.
The second rule protected me from the paper. It did not protect me from myself, which is the last third of this post.
What CodeNib claims about agent tokens
