I had an incident at work where an AI-powered agent made a change that passed CI and seemed entirely reasonable in the PR. The problem wasn’t just with the code—the issue was that we had never properly verified what the agent was actually allowed to do, which tools it could run, and where its authority ended. And to be honest, this incident completely changed my view of the review process!
I hadn't really thought about what the agent does behind the scenes. I mean, where exactly it sends the data or if it sends it at all. If a problem came up while checking the code or comparing something in the code, I’d report it as a bug with the agents and ask for it to be fixed! After the agents fixed the bug, I’d test it again and was satisfied with the result, but I didn’t notice what the agents were doing behind the scenes.
For most of us, code review means just one thing - someone reviews the changes in the code. One person made the changes, another reviewed them, and the reading was the control. Not the tests, not the linter, the reading.
That stopped being true on a lot of teams and I don't think most of them have said it out loud yet. Picture the pull request an agent opens a few hundred changed lines across a dozen files, a tidy description, green CI. The reviewer opens it and reads the description. Spot-checks the migration and the tests. Approves. Nobody read the few hundred lines. Nobody was ever going to.
