TL;DR — Constrained decoding and JSON schema enforcement guarantee that model output parses — they say nothing about whether the values are true, safe, or grounded in real system state. Treat structured output like you'd treat any untrusted client at an API boundary: schema validation is step one, not the whole job. The dangerous bugs left in production aren't malformed JSON anymore; they're well-formed lies.
Structured output was sold as a solved problem. Constrain the decoder to a grammar, force the model to emit valid JSON against a schema, and the parsing errors that plagued early LLM integrations disappear. They did disappear. But somewhere along the way, teams started treating schema-valid as a synonym for correct, and that substitution is quietly causing a new class of production bugs that don't look like bugs at all — they look like clean, well-typed data.
This is worth being precise about, because the word "type" is doing a lot of unearned work in how people talk about function calling and structured generation.
The Guarantee You Actually Got
Grammar-constrained decoding gives you exactly one guarantee: the output will conform to a shape. If your schema says status is an enum of three strings, the model will emit one of those three strings. If it says amount is a number, you get a number, not a sentence fragment. That's real, and it's valuable — it eliminated an entire category of glue code that used to exist purely to recover from malformed responses.
