Retrieval-Augmented Self-Recall — Part 5: The Gap Threshold That Didn't Transfer

Part 5 of Retrieval-Augmented Self-Recall. Code: RE-call. Part 4: the eval harness.

I shipped the gap_warning guard from Part 3 with a sensible-looking default: if the best cosine similarity is below 0.50, call it a probable gap and abstain. I tested it. It worked.

Then I swapped the embedding model. Every question that should have been refused came back as a confident answer.

No error. No crash. No failed test. The guard was still there, still running, still reporting that everything was fine. It had just quietly stopped being a guard.

Here's the mechanism, because it generalises well past my project, and there's a decent chance it's live in your RAG system right now.

Retrieval-Augmented Self-Recall — Part 5: The Gap Threshold That Didn't Transfer

Related reading

Retrieval-Augmented Self-Recall — Part 6: The Fine-Tune That Did Nothing, and…

Retrieval-Augmented Self-Recall — Part 4: Benchmarking Retrieval and Honesty

Retrieval-Augmented Self-Recall: The RAG Problem Nobody Talks About

Retrieval-Augmented Self-Recall — Part 3: Teaching RAG to Say \"I Don't Know\

Retrieval-Augmented Self-Recall — What the Comments Taught Me (RE-call v0.3)

Retrieval-Augmented Self-Recall — Part 2: Hybrid RAG on Nothing but Postgres

Related reading

Retrieval-Augmented Self-Recall — Part 6: The Fine-Tune That Did Nothing, and…

Retrieval-Augmented Self-Recall — Part 4: Benchmarking Retrieval and Honesty

Retrieval-Augmented Self-Recall: The RAG Problem Nobody Talks About

Retrieval-Augmented Self-Recall — Part 3: Teaching RAG to Say \"I Don't Know\

Retrieval-Augmented Self-Recall — What the Comments Taught Me (RE-call v0.3)

Retrieval-Augmented Self-Recall — Part 2: Hybrid RAG on Nothing but Postgres

Related reading

Retrieval-Augmented Self-Recall — Part 6: The Fine-Tune That Did Nothing, and…

Retrieval-Augmented Self-Recall — Part 4: Benchmarking Retrieval *and* Honesty

Retrieval-Augmented Self-Recall: The RAG Problem Nobody Talks About

Retrieval-Augmented Self-Recall — Part 3: Teaching RAG to Say \"I Don't Know\

Retrieval-Augmented Self-Recall — What the Comments Taught Me (RE-call v0.3)

Retrieval-Augmented Self-Recall — Part 2: Hybrid RAG on Nothing but Postgres

Related reading

Retrieval-Augmented Self-Recall — Part 6: The Fine-Tune That Did Nothing, and…

Retrieval-Augmented Self-Recall — Part 4: Benchmarking Retrieval *and* Honesty

Retrieval-Augmented Self-Recall: The RAG Problem Nobody Talks About

Retrieval-Augmented Self-Recall — Part 3: Teaching RAG to Say \"I Don't Know\

Retrieval-Augmented Self-Recall — What the Comments Taught Me (RE-call v0.3)

Retrieval-Augmented Self-Recall — Part 2: Hybrid RAG on Nothing but Postgres

Retrieval-Augmented Self-Recall — Part 4: Benchmarking Retrieval and Honesty

Retrieval-Augmented Self-Recall — Part 4: Benchmarking Retrieval and Honesty