This article was originally published on davidohnstad.net. I cross-post here to reach the Dev.to community.
We Built an AI-Powered Vendor Risk Tool Nobody Used
We spent fourteen months building an AI-powered vendor risk assessment system. The engineering team was proud of the model's recall rates. Security leadership presented it to the board as a competitive differentiator. Procurement got trained on the new workflow. Then we ran the usage analytics six months post-launch: 11% of vendor assessments actually used the AI scoring. The other 89% defaulted to the manual checklist we'd promised to deprecate. According to Gartner's 2024 Enterprise AI Adoption Survey, we weren't outliers — 72% of AI features shipped in enterprise compliance tools see adoption below 30% within the first year.
The problem wasn't the model. The precision was fine. The interface was clean. The problem was we never asked whether AI was the right tool for the decision we were automating. We assumed that because vendor risk could use AI, it should. That assumption cost us a year of roadmap bandwidth and created a product feature that procurement teams actively route around.
David Ohnstad has seen this pattern repeat across enterprise software implementations: teams shipping AI features because the technology exists, not because the business process demands it. The gap isn't technical capability — it's decision architecture. Most vendor risk workflows don't need probabilistic scoring. They need consistent application of known rules, audit trails, and clear escalation paths. AI introduces uncertainty into a process where certainty is the entire value proposition.
