Your AI agent can write code, summarize legal briefs, and book flights. But can it do those things correctly 100 times in a row without accidentally leaking sensitive data or going rogue on a prompt injection? That’s the question Amazon thinks the entire industry is getting wrong.
Bryan Silverthorn, director of Amazon’s AGI Autonomy research lab, is set to present a new framework at VB Transform 2026 that reframes the conversation around AI deployment. The core argument: enterprises aren’t holding back on AI agents because the technology isn’t smart enough. They’re holding back because it isn’t predictable enough.
The capability-reliability gap
Silverthorn’s presentation, titled “Closing the capability-reliability gap,” takes aim at one of the AI industry’s favorite measuring sticks: benchmark scores. Those EVAL scores that labs love to trumpet on launch day? According to Amazon’s framework, they’re essentially vanity metrics for enterprise buyers.
Amazon’s new framework explicitly prioritizes consistency, robustness, predictability, and safety over raw performance on standardized tests.
