discovery that changed how I think
about AI evaluation — and led me to
build an open-source testing framework.
─────────────────────────────────────────
Introduction
The Watermelon Effect: How My AI Scored 94% in Testing But Only 22.2% in Real Use
discovery that changed how I think about AI evaluation — and led me to build an open-source testing...
TL;DRAI
AI tutor ARIA scored 94% in automated testing but only 22.2% compliance in live use. Developer built BCT framework to test behavioral contracts under adversarial pressure, revealing gap between standard evaluation metrics and real-world promise-keeping.
1,444 words~7 min read
