Thirty seconds. That was my timeout. Why? It felt safe. It was not.
I watched a batch job die twice. First, requests failed before the server warmed up. Then, one hung request blocked everything behind it. My timeout was the problem. Not the model. Not the network. My guess.
A timeout is a latency budget. Guess it, and you pay twice. Too short means false failures. Too long means silent stalls. Retries only amplify both. I learned that the hard way in my last experiment.
So I stopped guessing. I measured. Here is the 20-minute calibration I now run against any free model server.
The Failure Mode Nobody Measures
