The tools built to measure how dangerous AI can be have stopped working. Frontier models outpace the benchmarks meant to gauge their hacking skills, Axios reports. That leaves regulators and security teams half-blind to what these systems can really do.
The timing bites. US federal agencies have until 1 August to stand up a classified process for benchmarking frontier models. The Financial Times reports the standards could land this week.
Saturated in months
Static tests age fast. Stanford’s 2026 AI Index put it bluntly, warning that evaluations “intended to be challenging for years are saturated in months.”
Older benchmarks set narrow puzzles. Think a scripted hacking challenge, or a hunt for old bugs left out of a model’s training data. Reasoning models such as Anthropic’s Mythos Preview and OpenAI’s GPT-5.5 now blow past them. The same leap also speeds up real attacks.
