Cross-posted from the IO reader blog, where the full version includes all 36 unedited transcripts side by side.
With gemini-2.5-flash scheduled to retire on October 16, we needed to audition replacements for a production workload that generic leaderboards say nothing about: twelve distinct "reading lens" prompts — literary close reading, Socratic questioning, EN→KO translation under a hard register constraint, vocabulary coaching — that our iOS reading app serves through Firebase AI Logic.
So we ran the audition the unfashionable way: one adversarial stimulus, every production prompt verbatim, three models, and a human reading + LLM judge (Claude Fable 5).
Method
Stimulus. The closing paragraph of Melville's "Whiteness of the Whale" chapter — chosen to break things: archaic vocabulary (subtile, palsied, charnel-house), one enormous periodic sentence, nineteenth-century natural philosophy, and a live metaphysical argument. If a model holds this paragraph steady through twelve different lenses, it holds almost anything a reader will meet.
