Test Impact Analysis is a simple promise: don't re-run the tests your change couldn't possibly have broken. Record which tests touch which code once, then use your git diff to run only the affected subset. Google and Meta do it internally; pytest-testmon does a local-dev version. I built an open-source one for pytest called tia — and the most important thing I learned had nothing to do with the algorithm.
It was this: the first time my benchmark looked great, it was lying to me.
A number too good to be true
To prove tia actually saved time, I avoided the trap of a toy demo where every test hits a unique function — that's rigged to look good. I pointed it at Flask, a real third-party suite, and replayed its last 15 real commits. For each one: record the impact map on the parent commit, then measure how much of the 483-test suite tia would skip for that change.
The result came back: median skip rate 73%. I almost wrote it down and moved on.
