LettucePrevent - Real-Time Prevention of Factual Hallucinations in RAG

Back to Articles

First of all, a huge thank you to my two supervisors Adam Kovacs and Gábor Recski!!

TL;DR:

LettucePrevent integrates a token-level detector directly into the generation loop with a custom LogitsProcessor, outperforming state-of-the-art hallucination detection models (HDMs) under streaming inference at lower latency.

Our regex-verifiable number detector achieves a >60% relative reduction in numeric hallucinations across all evaluated models with minimal latency overhead, ready to drop into any Hugging Face generate() call.

LettucePrevent - Real-Time Prevention of Factual Hallucinations in RAG

Related reading

LongTracer: Open-Source RAG Hallucination Detection Without LLM-as-a-Judge

0% vs 50%: Making a RAG Agent Refuse to Hallucinate

I Added a Verify Layer to My Local RAG to Catch Hallucinations. It Caught Me…

Building RAG that doesn't hallucinate

RAG - Hallucination Detection

AI agent codebase semantic RAG: AST for 30% Fewer Hallucinations