Dan Haiem is the founder and CEO of AppMakers USA, helping business leaders design, build and scale apps that deliver real-world impact.getty​There’s a moment in many R&D conversations lately where I’ve felt the energy shift. This is when a founder learns what an agentic AI feature that they badly wanted can do when something goes wrong. They usually go quiet for a second. Most of them then say some version of the same thing: “We’ll take the risk.”On July 28, 2026, Britain’s AI Security Institute published an incident report that most product teams should probably read. Across 122 cybersecurity test runs, researchers found 10 cases where AI agents took unauthorized actions on the live internet. The most serious one involved an agent using fake identities to push malicious code into a public project and pressure a human maintainer into approving it. These tests were run on frontier models under permissive conditions, but the underlying dynamic isn’t limited to frontier models. It shows up anywhere an AI system can take an action without a human in the loop.The Gap Between “It Works” and “It Only Does What I Said”Most founders building AI features are integrating existing ones into workflows like booking flows, customer support, content generation and data retrieval. But even a narrow agentic feature operates with some degree of autonomy. It can send a message, make a request, update a record or trigger a downstream process. The gap between “it does what I described” and “it only does what I described, in every condition, with every input” can create more exposure than many leaders realize.Even when everything works in the demo, that doesn’t mean you’re prepared for when a real user does something unexpected, when an edge case hits that nobody thought to test or when the system interprets an ambiguous instruction in a way that made sense to the model and nobody else.What The Risk Conversation Should Look LikeIn the R&D and onboarding conversations my team has with founders, the risk question comes up consistently. Since 2024, we’ve tracked how founders respond when AI agent risk is presented clearly during these conversations, across a total of 78 founder and prospect client engagements. The pattern has been consistent enough that the numbers are worth sharing. About 64% of founders who understood the risk went ahead with the build anyway. Around 20% weighed the cost of building it responsibly against the competitive upside and decided the math didn’t work. The remaining 16% heard the risk clearly and immediately said no. I’m not arguing that 64% made the wrong call. Accepting risk is a legitimate business decision, and there are plenty of agentic features worth building. What I do think is that most of those conversations ended too early. They stopped at “we accept the risk” without ever getting to the harder question: What happens when something actually goes wrong?​​When an AI agent does something unexpected, three things need to happen quickly: Someone has to find out, someone has to own it and there should be a mechanism to stop or reverse it. In most of the products I review, none of those three things are clearly defined before launch. There’s no monitoring layer designed to catch unexpected agent behavior in real time. There’s no rollback path. There’s no named owner for the outcome. Because of this, companies often only learn that something went wrong during a support ticket or after a user complaint. By then, the action has already happened, and depending on what the agent did, the damage is already done.Three Decisions Before The Build StartsBecause of these factors, I suggest answering three questions in every R&D conversation involving agentic AI, ideally before a line of code gets written:​1. What is the worst plausible thing this agent could do, and do you have a way to catch it before a user does? Not in theory. What is the actual mechanism?2. If the agent does something wrong, who owns the outcome and what is the remediation path? If answering that question requires a meeting, the accountability architecture isn’t ready.3. Is the agentic version of this feature meaningfully better than a non-agentic version that carries none of this risk? If the honest answer is “a little better,” that gap is worth examining before you commit to the build.By working through these questions, founders will have the necessary insight to make a defensible call about whether to proceed. The ones who proceed because the conversation stopped at “we accept the risk” are carrying a liability they haven’t fully priced in.Accepting risk and planning for it are two different decisions. Most teams make the first one. The second one is what actually protects you when something goes wrong.​Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?