We're changing the default solver model in our eval harness from Claude Sonnet 4.6 to GLM 5.1. This is the default we provide to everyone running evals on the platform. For most of the work the harness does, a frontier model gives you the strongest possible signal. However, that's more signal than the job needs and the difference is where eval budgets quietly leak. The question that decides how much you should be paying is whether a given eval run is measuring the model or measuring the skill.
The principle behind it: specify the model only when you care about the model. When your eval exists to answer "does this specific model ship well?", you have to run that exact model. When it exists to answer "does this skill improve agent behavior, and has anything regressed?", you don't need a specific model, you need a representative one.
We put this to the test on our own skill-evaluation harness and validated GLM 5.1 against Sonnet 4.6, the model it replaces as the default. We lost almost none of the signal skill authors rely on, and the eval bill went down. This post is the reasoning behind the switch, and a framework you can apply to your own eval stack.
Two questions, one eval harness
Our harness runs a large skill-evaluation suite: roughly 500 skills across about 850 tasks, each run twice, with the skill and without it. We score three things: instruction following (did the agent do what the skill tells it to do), task completion (did it reach the goal), and an overall blend weighted toward instruction following.
