If you're building applications with OpenAI, Gemini, or LangChain agents, you already know the pain: Large Language Models are unreliable.

You ask for a JSON response. You set up a strict parser like Pydantic or Marshmallow. But then:

The LLM cuts off mid-sentence because it hit the token limit.

The output has a missing closing bracket }.

The LLM outputs Python-style single quotes ('id') or True instead of standard double quotes and true.