Artificial Analysis has rolled out Harvey LAB-AA, an independent benchmarking framework that tests AI language models on 120 private legal tasks spanning 24 practice areas. The top performer, Claude Fable 5, managed an all-pass rate of just 14.2%.
What Harvey LAB-AA actually measures
Harvey LAB-AA isn’t your typical chatbot leaderboard. It’s an independent implementation of Harvey AI’s Legal Agent Benchmark, which was originally open-sourced on May 6, 2026. The full benchmark includes more than 1,200 tasks evaluated against over 75,000 expert-defined rubric criteria.
The “all-pass” grading standard is deliberately brutal. A task only counts as successful if every single rubric criterion is met. Not most of them. Not the important ones. All of them.
Artificial Analysis runs its evaluations using a system called the Stirrup harness, applied to Harvey’s private task subset. The framework reports both task completion rates and specific criterion performance metrics.
