Most "AI chat widget" tutorials assume you're calling out to an LLM API. This post is about the version where you don't -- where the model runs in the visitor's own browser, the cost per conversation is exactly zero, and nothing about what someone asks ever leaves their machine.
The problem with the obvious approach
A server-side LLM call is simple to build and expensive/slow/privacy-leaky to run at scale on a marketing site that gets real traffic. For a small business's website, "every visitor's chat questions get logged on someone's server" is also just a worse default than it needs to be.
The 3-tier fallback
Chrome's built-in Prompt API (Gemini Nano) -- when the browser supports it, this is a real, conversational model running locally, grounded in a small knowledge base specific to the site.
