Most tests are cheap. You click a button, you assert something changed, you run it a thousand times and nobody notices. Testing an AI product is different, because the interesting behaviour comes from a model, and every time you trigger it you pay for it.
We ran straight into this while building a course product on top of The AI Platform. I want to walk you through how we ended up testing the whole chat flow end to end with Playwright.
First, what the platform is
The AI Platform (TAP) by Zephyr Cloud is a desktop app where teams collaborate with AI specialists in channels. Think Slack, but some of the people in the channel are AI agents you can talk to, mention, and hand work to.
You create a channel, mention a specialist, and it responds in the conversation like any other member would. That chat surface is the heart of the product, so it is also the thing we most need to trust.
