31st July 2026 - Link Blog
smevals - a small eval suite for evaluating models, prompts, and harnesses. I've been working with Jesse Vincent's Prime Radiant applied AI research lab building out this evals framework to help answer questions about the capabilities of different models.
The result is smevals, a new tool for running small eval suites across different model configurations and grading the results.
This blog entry describes the tool in detail. Here's the 10 second version:
Tell your coding agent to run uvx smevals docs to learn the tool (this outputs the README)
