I Almost Shipped a RAG Assistant That Lied About APIs That Don't Exist

I wrote this on X a few weeks ago:

I just had a very bad reminder as to the fact these LLMs are statistical parrots, I let it write code I normally wouldn't trust it to write (infra code, lots of unique behaviours) and damn

I wasn't talking about my own project when I wrote that. Then StacksNG proved me right, on its own corpus, in a hackathon I'm trying to win.

Ask my RAG assistant to verify an Interswitch webhook signature, and it didn't say "not in my knowledge base." It wrote a full authentication flow — real-looking endpoint, real-looking headers — and cited a source URL. The URL wasn't in my corpus. It wasn't anywhere. The model invented a citation for content it also invented, with zero hedging.

I'm building StacksNG for the Africa Deep Tech Challenge 2026 — an offline coding assistant scoped to the African fintech stack: Paystack, Flutterwave, Monnify, Termii. Before I submitted, I ran a 20-prompt adversarial batch against my own pipeline. Category A (in-corpus baseline) and D (phrasing brittleness) came back clean. Category B — five prompts asking about payment providers I deliberately never scraped into the corpus, Kuda, PalmPay, Interswitch, Paga, OPay — did not.