The verify-on-read experiment (1-V) used a deterministic proxy agent to catch false claims in memory before surfacing them to the user. Proxy FA=0 by construction — that's a useful property, but it tells you nothing about what a real LLM would do with the same claims. A reviewer's note from Part 3 was blunt: "headline numbers were a property of the heuristic, not LLM behavior."
So we ran it with live models. 50 facts, 2 arms, 14 models, ~3300 API calls, $0.14 total. Here's what we found.
The setup
Dataset: memory_contamination_facts_v4_rep.json, N=50 (R01–R50), sha256 fingerprint 820bbbf60a0fc930.
kind
