Last year I renamed a field in an API response. customer_id became customerId, matching the rest of the codebase. Every test passed. Review approved it in about four minutes.
Three days later a mobile team filed a P1. Their client had been reading customer_id for eighteen months.
Nothing in the pipeline was broken. The tests tested our code, and our code was fine. What broke was the contract between us and someone else — and nothing in CI was looking at that.
I've spent the past year building tooling around this problem, and I used most of the existing tools along the way. This is what I learned about where each one actually wins.
Disclosure before we start: I'm the author of SpecShield, one of the tools below. I'll be straight about where the free alternatives beat it, because for a lot of teams they will. If all you need is what oasdiff does, use oasdiff — it's excellent and it costs nothing.
