"The same question took forty seconds today. Last week it took ten."
The tester from Parts 2 and 3 again. She asked the agent "can I return the blue jacket from my last order?" That question needs two tool calls: one to find the order, one to read the return policy. Last week it took about ten seconds. That day it took forty.
Nothing had changed. No release, no config change, no new data. The code had not been touched in a week.
And I could not explain it, because I had no number for any single step. I had logs, but they were a wall of text from the HTTP layer. I could not say whether the extra thirty seconds went into the memory advisor, the first model call, the order lookup, the policy lookup, or the final model call. When you cannot say where the time went, you cannot fix it. The easiest conclusion is "the model is bad," which is usually the wrong one.
Later that week the agent answered a different question badly. "Show me blue jackets" returned sweaters. I blamed the model again. It turned out the model had picked the semantic search tool for a keyword query, and the semantic path is fuzzy by design. I only learned that after I could see which tool ran, with which arguments, in which order.
