Approval testing is an older idea than snapshot testing and a more honest description of what a golden-file suite for model output does. The test does not know the right answer. A person does, and the file records that they looked.
The received and approved pair
The mechanism is two files per case. The run writes what it got to a .received file. If a .approved file exists and the two match byte for byte, the received file is deleted and the test passes. If they differ, the test fails and a reporter launches — a diff tool, an editor, or in CI a plain textual diff — showing the human exactly what changed. Approving means copying received over approved and committing it.
In ApprovalTests.Python the call is verify(result), imported from approvaltests.approvals, and the file names are derived from the test’s identity. The library exists in Java, .NET, C++ and JavaScript with the same two-file protocol. Nothing about that protocol is specific to language models; it was built for legacy code whose output nobody could specify in advance, which is a strikingly good description of a prompt.
# tests/test_summary.py
