Pieter van Schalkwyk, CEO at XMPro, specializing in Multi-Agent Generative Systems for industrial operations.getty​Insurers have started backing away from covering artificial intelligence (AI), and the way they are doing it tells you more about the real risk than any vendor pitch will.In 2025, as reported by TechCrunch, large carriers including Great American, Chubb and W.R. Berkley asked state regulators for permission to carve AI-related claims out of corporate policies, with some exclusions reaching into directors and officers and errors and omissions cover. The Insurance Services Office​ also issued a standard generative AI exclusion, allowing carriers to add to commercial general liability renewals at the start of this year. Underwriters keep giving the same reason. They cannot price what they cannot explain, and a model that produces an output nobody can account for leaves them holding an open-ended bill when that output causes harm.The people who price risk for a living have looked at a class of decisions your business is starting to automate and decided they would rather sit it out. Compliance frameworks spent three years circling the governance question, but the insurance market answered it with a number, and the number was "we would rather not."I work on systems where AI does not sit beside an operator offering suggestions. These agents adjust a process, trigger maintenance or coordinate equipment inside plants where a bad call carries physical consequences and sometimes lives. The underwriter's worry stopped being theoretical for me a long time ago. On those sites, the first question is never whether the model is accurate, but whether we can reconstruct after the fact exactly why the system did what it did and whether we can prove it stayed inside the limits we set.​Plenty of AI deployments cannot do either, and that is the precise gap the carriers are now declining to insure.What Accountable AI Actually Looks Like​Most companies will read this as a coverage problem and hand it to the risk team.Instead, it belongs with the people who built and deployed the system, because what makes a decision insurable gets settled in the architecture long before anyone reads the policy wording. Start with the record. Any consequential action an AI takes should leave a durable trace: what it knew, what it weighed, what rule or objective drove the choice and what it actually did. A line in a developer log does not count. The trace has to be something an auditor, a regulator and your own general counsel can sit down and read months later. Without it, you do not have evidence of a sound decision. You have your word.Then there is the boundary, which is the piece that teams consistently underestimate. Let the model reason wherever it likes, but it should be physically incapable of acting outside the operating limits you have encoded, however persuasively it talks itself toward the edge. An AI that can argue its way past a safety interlock is the open-ended liability the carriers are running from, described almost exactly.​The harder change is where the governance actually lives. A review board that meets once a quarter cannot supervise the choices a model makes in milliseconds. The constraints your legal, safety and finance teams care about only do any work if they are wired into the point where the decision happens, so a non-compliant action gets blocked before it executes rather than investigated after it costs you. A binder on a shelf has never stopped a model mid-action.​The Price Of Lacking Governance​I would not wait for the insurance market to settle before moving on any of this. The exclusions are spreading faster than the specialist policies meant to backfill them, and the exposure lands on the board whether the cover does or not.​Before your next renewal, it is worth pinning down a few things. Find out if your current policy quietly stopped covering anything because the wording changed and nobody phoned to tell you. Then, for any AI making decisions that matter, ask whether you could defend one specific decision to an outsider a year from now. And confirm you can prove, with something that outsider would accept, that the system cannot step beyond its limits.The carriers have already moved. By walking away from decisions a company cannot explain and contain, they have drawn a line in the sand. That is worth sitting with, because it is the first time anyone has attached a real price to running AI without governance. The companies that treat the price as a warning will end up in a very different place from the ones still assuming the cover shows up on its own.​Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?