Every time a new model ships, the same ritual: change the model string, run the agent, read a few replies, they look good, ship it.
The replies are the one part of an agent that almost never breaks visibly. What breaks is behavior. A tool call quietly disappears, an argument drifts, a refund amount loses its decimal point, and the agent keeps talking like everything is fine. I learned this the hard way when a swap made my agent stop calling cancel_subscription while it kept telling users their subscription was cancelled.
With a new frontier model out this week, a lot of model strings are about to change. This is the check I now run before any swap. It takes about 20 minutes and produces a real diff instead of a vibe check.
1. Record a baseline before you touch anything
This is the step you cannot recover later. Once you swap, the old behavior is gone.
