A friend who runs security at a mid-sized fintech called me three weeks ago. Her board had asked a question her team couldn't answer in under a week: "How many models are running in production, and which ones touch customer PII?"
She sent me her first draft of the answer. It was a spreadsheet. Forty-seven rows. The columns were Model Name, Owner, Risk Tier, Data Classification. Most of the cells said "TBD" or "see Notion." Two rows were duplicates of the same model deployed in different namespaces. One row was a model that had been deprecated six months earlier but was still receiving traffic because a forgotten Lambda kept calling its endpoint. Another row didn't exist on the spreadsheet at all — it was a fine-tuned Llama variant running inside a vendor's SaaS that her data science team had embedded via SDK. Nobody had told security.
The part that bothered her wasn't the spreadsheet. It was what the spreadsheet implied. Her org had stood up an AI governance committee, hired a head of responsible AI, and bought a "model risk management" platform from a name-brand vendor. And she still couldn't answer the board's question without two weeks of Slack archaeology. The vendor's tool had a beautiful dashboard. It just didn't know about half the models, none of the agents, zero of the MCP tools, and obviously nothing about the prompts and retrieval indexes hanging off them.
