The last lesson was about validating what a model hands you. The story behind it: a set of prompts that had returned real, criteria-matched vendors for weeks came back in staging with placeholder junk, literally the words Vendor A, Vendor B, Vendor C. So I built the validation layer, and the last gate in it is a model checking a model.
Then FromZeroToShip asked three questions in the comments, and all three were about the gate rather than the model. That's the harder thing to look at, and I hadn't written all of it down. Here's the long version.
What was on the fail list that I hadn't already been burned by?
More than the question assumes, and not because I got clever about imagining failures.
The placeholder output changed what I do with a failure. I stopped fixing the instance and asked what class it belonged to, and that class is a lot wider than "the model emitted example data." It's a suggestion that looks fine and isn't usable. Two of those I had never hit went in on the back of it:
