In my last post I said that for normal HTML pages, element-based automation (find / read_page) beats coordinates every time. This post is about the apps where that advice is useless.
Flutter Web apps. Canvas-rendered editors. Every button and panel you can see on screen doesn't exist in the DOM — it's all pixels painted onto a single canvas. find returns nothing. read_page's accessibility tree is effectively empty.
I got Claude to drive the Rive editor (an animation tool built with Flutter) all the way through selecting assets and exporting them. Here's the procedure that survived contact with reality.
Step zero: confirm you're actually in this situation
Coordinate automation is fragile, so you should only accept it after ruling out the alternative. The test is quick: run read_page. If the visible UI has almost no corresponding nodes, you're looking at a canvas-rendered app, and coordinates are the only interface you have.
