A measured study of multi-provider LLM consensus as a locator-healing signal — and why Automation Sandbox keeps the LLM out of the decision. Originally published on the project site.
TL;DR
When a UI refactor deletes an element, a self-healing engine must decline and ask a human — not latch onto a neighbouring button. I tested whether asking several independent LLMs and requiring them to agree can act as that "the element is gone" detector.
Across four live multi-provider runs (2026-08-16 to 2026-08-18, 133 usable scenarios):
On elements that still existed (moved/relabelled), unanimous provider agreement was correct 52 / 52 times.
