Benchmarking AI models on coding puzzles and trivia is easy. Benchmarking them on whether they can actually do a job is much harder. Artificial Analysis just took a serious run at the latter.
The independent AI research and evaluation firm has launched EnterpriseOps-Gym-AA, a platform designed to measure how well large language model-based agents complete real, multi-step tasks inside live enterprise systems. Think filing an IT ticket, resolving a customer service issue, or navigating HR workflows, end-to-end, with no hand-holding.
The early results are telling. Claude Fable 5, Anthropic’s latest model, leads the inaugural leaderboard with a 51.1% task success rate under oracle tool mode, a configuration that includes adaptive reasoning and fallback mechanisms. That number sounds modest until you see where the field was sitting before: the original ServiceNow paper that inspired this style of evaluation reported a top score of 37.4% for models including Claude Opus 4.5, during evaluations conducted in March 2026.
Going from 37.4% to 51.1% in the span of a few months is meaningful progress. It is also a reminder that even the best model in the world still fails nearly half the time on tasks a competent office worker handles before lunch.
