There is a persistent myth that to build a worthy code assistant, you absolutely must use GPT or Claude. This is false. You don't need a 1-trillion parameter model. You need a small local model and extremely rigorous engineering around it.
This is the direction history is taking for companies. As Mark Zuckerberg mentioned, the future isn't a single omniscient model, but "every company having its own specialized AI". And this specialization necessarily involves fine-tuning and local deployment (or on sovereign servers) to guarantee data security.
The thesis behind the construction of Vibrisse Agent can be summed up in one sentence: Small models, Great tools.
In this article, I will detail the technical stack and concrete engineering solutions I implemented to tame a local model and make it reliable in production: LangGraph, Ollama, FastAPI, React (no build step, with embedded custom CSS), all running on a machine with 32 GB of RAM.
For the curious who want to run the agent on their machine right now:
