Last week I was testing an AI chat feature on my iPhone 14 (iOS 17.5, React Native 0.74, Hermes). The flow looked fine on Wi-Fi. Then I walked into an elevator mid-stream, the radio dropped, iOS suspended the app, and when I came back the conversation had silently lost the last two turns. No error. No retry. Just a gap in the transcript that only the user would notice.
The problem with testing this class of bug is that I don't control the model backend. I can't tell a hosted LLM API to hang for 40 seconds, drop a socket mid-response, or return a truncated chunk — and those are exactly the conditions my app fails under. So I started routing the app through a relay server I can kill, and suddenly reconnect behavior became a benchmarkable property instead of a vibe.
This post is the harness, the test matrix, and how I run it. Everything below is a reproducible setup, not a finished benchmark report — where I expect a result rather than having measured one, I say so.
Why a relay you control
Point your mobile app at a thin relay in front of the real model API. The relay proxies requests but lets you inject:
