Originally published at dmitriiev.dev
Here is the rule the whole thing is built on: every stage is checked by something that did not produce its output. A parser, a different model, a real exit code, a person. No agent gets to declare its own work done.
That is not an AI idea. It is one of the oldest in engineering under a newer name: the thing that makes something should not be the thing that certifies it. Compilers, CI, code review, mutation testing all run on it. Producer is not validator.
Most of the 2026 wave of AI in testing forgets it. The pitch is to let an agent write and run your tests. A lot of what ships drops a model into the hot path of CI: the same kind of system that wrote an assertion also decides, on every build, whether that assertion is reasonable. That is a closed loop. A model is good at inventing a plausible test and much worse at catching its own wrong assumptions, and letting it grade itself just lets it be confidently wrong twice in the same direction. Writing tests is the cheap part an LLM is genuinely good at. Proving they are worth running is the work, and it takes someone other than the writer.
So the writing here is almost incidental, and every artifact a model produces gets handed to something else to check. None of this is a whiteboard design; it comes out of maintaining real Playwright suites, not out of a diagram. The whole thing is public, demo app included, and runs end to end in a few minutes: ai-qa-pipeline.
