The demo always works. You prompt the model to "return JSON with these fields," it returns clean JSON, you parse it, everyone claps. Then you put it behind real traffic and real user input, and a week later something downstream throws because a field you assumed was a number came back as the string "two" and nothing validated it on the way in.
This post is about the layer between "the model usually returns the right shape" and "the rest of my system can depend on the shape." That layer is not glamorous and it is most of the work. The examples use a fictional telecom assistant that turns a customer's freeform message into a structured intent, but the pattern is the same whether you're extracting invoices, classifying tickets, or routing a support request.
what actually breaks
When you rely on an LLM to produce structured data, the failures are not the ones you plan for. In roughly the order they cost me time:
Malformed output. The model wraps the JSON in a code fence, or adds a friendly sentence before it, or trails off mid-object under load. Rarer now with structured-output modes, still real.
