Former lawyer Dan Hauck (Chief Product Officer, NetDocuments) is passionate about delivering enabling technology to the legal industry.gettyEvery few months, a new AI tool promises to transform legal work. The demos are impressive. The capabilities are real. And yet lawyers keep describing the same frustration that the AI doesn’t quite understand the work, with lawyers' feelings about AI becoming less positive between 2025 and 2026, according to a Law360 Pulse survey. As the CPO of a legal document management system, I've found that many legal leaders try to solve this by addressing the models themselves when the problem is actually the context beneath the models. Even as models have improved over the past few years, legal AI fails less because it's "wrong" or "hallucinating" and more because it doesn't understand the work it's been asked to help with. It doesn’t know the matter history. It doesn’t know which precedent actually matters. It doesn’t know who negotiated the key clause or why one document matters more than another.In the legal industry, those details are the work itself, not background information. The Context Problem Hiding In Plain SightFor decades, law firms have lived with a knowledge problem that everyone accepts as normal. A new associate joins a complex matter and spends days reconstructing context that already exists somewhere in the firm. A partner asks whether the firm has handled a similar issue before, and the answer depends on who happens to be in the room. A senior lawyer retires, and decades of judgment, precedent and practical wisdom become harder to reach.AI changes the equation fundamentally. Lawyers are no longer the only consumers of legal knowledge. AI assistants, agents and emerging autopilots now need to work with that knowledge, too—and an agent that can’t reach the right context is working blind.What 'Context' Actually Means In Legal WorkLegal work is language, not clean database fields. A clause can look standard until the exception changes the risk profile entirely. A matter can turn on the relationship between a pleading filed three years ago, a deposition taken last month, an email chain and a prior decision that established the firm’s position. The most valuable context is rarely in any single document but rather exists in the connections.This is why simply giving AI access to more documents doesn’t solve the problem. Gartner has identified "context engineering" as a strategic priority for AI leaders, recognizing that model quality matters less than the quality of context delivered to that model. Foundation Capital has argued that context graphs represent the next defining shift in enterprise AI. In other words, having the biggest models won't solve the challenges with AI unless the model knows why decisions were made.In legal, that insight hits particularly hard. The decisive question for any legal AI agent isn’t whether the model is good but whether the agent can reach the right context, under the right governance, at the right time.Why Governance Can’t Be An AfterthoughtIn most industries, context and governance are separate concerns. In legal, they’re inseparable. AI that can surface the right information but ignores an ethical wall is dangerous. Firms operate under a web of client restrictions, matter-level policies, outside counsel guidelines and jurisdiction-specific requirements that change continuously.The architecture matters here more than most people realize. If an AI tool builds its own copy of a firm’s data, governance becomes a synchronization problem. Every permissions change, ethical wall or client policy update has to be reflected somewhere else. And in legal, eventual consistency isn’t good enough.The right approach keeps context close to where the work already lives, in the system of record, connected to the permissions and policies that govern the work itself. That’s the only way an AI agent can know not just what information exists but also whether this user, in this matter, at this moment, is allowed to use it.What It Actually Takes To Build ThisScale makes it harder. Large law firms may have millions or hundreds of millions of records, and permissions and ethical walls continuously shift. Managing this amount of data involves building a system that extracts structure from unstructured legal language, indexes content for both exact and conceptual retrieval, connects records across matters and communications, enforces governance in real time and makes the result available to both lawyers and agents without creating uncontrolled copies of firm data. That is genuinely hard engineering.Most importantly, this kind of foundation has to be built into the core of the platform. You cannot retrofit context into a system designed to store documents. It has to be the design.What Changes When Context Is SolvedWith a legal context graph, a second-year associate staffed on a contract dispute doesn’t spend the weekend reconstructing the matter from scratch. She opens it and sees the picture that used to live only in a partner’s head: parties, timeline, key documents, claims, recent activity and the prior work the firm has already done on similar issues. She still exercises judgment. She just starts from a real foundation instead of a blank page.A search for relevant precedent returns what the lawyer was thinking of, not just what matched the exact phrase she happened to remember. An AI agent drafting a first pass works from the firm’s actual institutional knowledge, not a generic model of what contracts usually look like.The goal isn’t to replace legal judgment but to reduce the friction around context so lawyers can spend more time on the judgment, advocacy and strategy that actually requires them.The Race In Legal AIThe legal AI market is moving fast, and different tools are beginning to converge.Better models help. Better interfaces help. But the quality of the work AI produces will ultimately depend on the context layer underneath.Many of the challenges of legal AI won’t be solved by the tool with the biggest prompt box. Instead, the frustrations can be addressed by delivering the right legal context, with the right governance, when the work is happening.That foundation is what everything else depends on. ​​Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?