TL;DRAmazon’s security VP says human-in-the-loop AI governance fails fast because people stop paying attention. Google, Microsoft, and IBM agree.
Amazon’s security leadership is arguing against one of the most widely accepted principles in AI governance. Eric Brandwine, VP and distinguished engineer at Amazon Security, told The Register that human-in-the-loop oversight is not the gold standard companies think it is.
“Humans are not terribly consistent,” Brandwine said. “Human-in-the-loop isn’t necessarily the gold standard.”
His reasoning draws on a concept he has been talking about since at least 2017, when he gave a talk on normalization of deviance at AWS re:Invent. The term describes what happens when people in an organization take shortcuts over time, and nothing catastrophic results, so the deviant behavior becomes the new normal.
The 💜 of EU techThe latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!Brandwine illustrated the point with emergency rooms. On a nurse’s first day, every alarm triggers a response.
