Day 10. OCR and template matching hit their limits. UI hierarchy inspection might be the real answer.
Nine days ago, I was proud of my screenshot-based vision system. ML Kit for text. Template matching for icons. A clever fallback chain that worked most of the time.
Today, I'm ripping most of it out.
The Breaking Point
Last week, I tested the agent on a friend's phone. Template matching failed. The same icons I cropped on my device didn't match on his—different screen density, different rendering, different pixel arrangement.
