Running large language models typically involves cloud API endpoints, paying per token, managing rate limits, and transmitting sensitive user queries across the network.
With Chrome's Built-in AI initiatives (via the WICG Prompt API and Gemini Nano), browsers can now execute small language models (SLMs) directly on the user's hardware. Inference happens entirely on-device: zero API keys to expose in frontend bundles, zero cloud infrastructure bills, zero network latency after weight loading, and complete client-side data privacy.
This article walks through the technical mechanics of integrating Chrome's Prompt API into a modern JavaScript/TypeScript application, including capability detection, session lifecycle, streaming token consumption, in-context grounding, and deterministic parameter tuning.
1. The Prompt API Architecture
Chrome exposes on-device capabilities through the window.ai namespace (and globally as LanguageModel). Under the hood, Chromium interfaces with an optimized on-device foundation model (Gemini Nano) managed by the browser's Optimization Guide component.
