One pass of my eval bills $9.14 on the API and $0 through the CLI

One pass of my board eval bills $9.14 on the Anthropic API. Through Claude Code it bills $0. Same model, claude-opus-4-8.

That is 27 calls, and it is not an estimate. The CLI prints a total_cost_usd in its envelope: what the run would have cost on the API. It bills the subscription instead, so the number is a receipt for money nobody spent.

The switch fixed something better than the bill

Running with --output-format json and --json-schema rides the same structured-output machinery the API does, an internal forced tool call. Format reliability on my suite went from 7 out of 15 to 15 out of 15.

The schema needs relaxing first: strip pattern, minLength, maxLength, minItems, maxItems, format and the $schema meta-ref, because the CLI validator rejects draft-2020-12. The strict version stays in Zod on the caller side, so nothing is actually loosened, the validation just moves to where it can run.