Gaurav is a Software Engineer at Walmart Global Tech. He plays a key role in building Agentic AI experiences for Marketplace sellers.gettyWhen an embedded AI assistant disappoints in production, the postmortem almost always blames the model. The model is rarely the problem. The real culprit is context architecture. In a multi-tenant SaaS product, the cost multiplies with every seat you sell.​Picture a controller logged into the finance dashboard, staring at 80 rows of invoices, clicking the AI button and typing, "Which of these have unmatched POs?" The assistant responds with a confident answer about a row that is not on the screen. Or worse, it replies, "I can't access your invoice data." Either way, the user closes the panel and stops trusting the feature. The model was not wrong. It never saw what the user was looking at.Gartner predicts that "40% of enterprise apps will feature task-specific AI agents by 2026, up from less than 5% in 2025." They are making representation decisions right now that will show up on your gross margin report in six months. In my work building the context layer for an AI assistant embedded in a B2B SaaS product, I've watched teams nail model selection and ship a broken cost structure. Anthropic Engineering calls this "context engineering": the practice of deciding what information an AI model receives at each step. Because every token adds inference cost, those context choices also become gross-margin decisions.​​Why The Representation Layer Is Where Your Margin LeaksEvery embedded agent has three choices for what it sends the model: raw HTML, a screenshot or a semantic snapshot.Raw HTML is the default. It feels complete, requires no preprocessing and is a trap. A typical enterprise dashboard page could run to roughly 150 KB of HTML, approximately 37,500 tokens, in my experience, and most of it is CSS class names and noise the model has to wade through before finding anything useful.A semantic snapshot is what remains after you strip the markup and styling and keep only the meaning: the headings, the table rows as structured records, the buttons as named actions and the current filter selections as explicit state. It is the page as a screen reader would describe it, except the reader is an LLM. A semantic snapshot produces 750 to 1,250 tokens. Anthropic's engineering team documented cutting an agent workflow from roughly 150,000 tokens to 2,000 by changing the representation, not the model."Which of these invoices have unmatched POs?" requires the agent to read the screen. In one mid-size SaaS deployment I've observed, defaulting to raw HTML pushed inference cost-per-active-user from $0.40 to $1.10 per month within eight weeks of general availability. The agent worked. The unit economics did not. The gross margin lift the feature promised was entirely consumed by the context tax before the team knew it had a name.Three Architecture Decisions Your Team Is Making Without YouThree decisions are already made before your team ships, whether they surfaced in a design review or not.1. First, what representation feeds the model. Raw HTML, screenshots and semantic snapshots carry 30 to 50 times cost differences between them, with accuracy differences that run opposite to what intuition suggests. Research from Amazon Science found that AgentOccam, which relies on a structured accessibility-tree representation rather than vision-first observations, outperformed the previous state-of-the-art by 29.4% on the WebArena web-agent benchmark. Choosing raw HTML because it requires no upfront work is not a technical decision. It is a margin decision made by default.2. Second, when context is captured. If your agent captures page state at load time, it captures the loading skeleton, not the actual data the user is looking at. I've seen agents answer confidently from a page of loading skeletons; the snapshot fired 1,500 ms before the API responded. Capturing on send, not on load, eliminates an entire class of hallucination.3. Third, what does not get sent at all. Not every query needs page context. A user asking how to bulk-update a setting needs documentation, not a snapshot of their current data view. In my experience, a lightweight classifier that routes before retrieving can eliminate 70% to 85% of snapshots, and these are tokens you never spend.Capture context at send time, not at load time. Use the semantic snapshot, not the raw DOM. Route before you retrieve.What the user has been doing in the session, not just what they're looking at, is a separate context layer that most teams haven't scoped yet. A user who clicks an item, reads a detail panel, then asks "What's wrong with this one?" is asking about something the agent never saw.​Why Your Customers Pay For Every Token You WasteMost writing on enterprise AI agents comes from an internal IT vantage: the agent that helps employees file tickets, query the data warehouse and navigate internal tools. That agent's blast radius is bounded. You tune and redeploy.Your embedded agent runs inside your product, for your customers, billed at your cost. Every inefficiency multiplies across your entire customer base at once. A three-times token overrun is not a next-sprint bug. It is a cost structure change affecting every seat you've sold. ​Chroma's 2025 research evaluated 18 frontier models and found that performance consistently degraded as input context grew, with substantial accuracy losses on several long-context tasks compared with focused prompts. The context tax is not a cost problem alone. Accuracy degrades with every UI layer you add.Internal IT teams can absorb a bad context architecture. B2B SaaS vendors bill for product, not for infrastructure inefficiencies.The Window Is Shorter Than You ThinkOver the next 12 to 18 months, embedded AI standards may shift from "Does it work?" to "Does it work without inflating your cost structure?" Teams treating context architecture as a product strategy decision today will have a year of cost-per-query data when that question hits the board. Those that defaulted to raw HTML will be explaining why the AI feature meant to expand gross margin is compressing it.The model is not the bottleneck. The representation layer is. And that is an architecture decision, not a vendor decision, which means it is yours to make.​​Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?