I started prototyping an AI assistant for CAPA triage because our backlog was burying the QA team. The idea felt obvious: have a model read an incoming nonconformance or complaint, suggest probable root-cause categories, surface related docs from the technical file, and propose a draft corrective action for a human to edit. It sped up screening in a sandbox — until the uncomfortable question landed: how do we validate this so a regulator (or my internal auditors) can accept it?
If you work under 21 CFR 820 and ISO 13485, you already know CAPA is not a place for unproven gamble: 21 CFR 820.100 expects documented procedures and verification of effectiveness; ISO 13485 expects controlled corrective/preventive processes. But those standards are written for human processes and classic software validation. AI changes the shape of "verification" and "evidence."
Below are what I learned by prototyping, plus a pragmatic validation pattern we used to move from experiments to a controlled assistive deployment.
The core problem: what is "validated" when a model is probabilistic?
Traditional software validation is: input X -> expected deterministic output Y. With an LLM, output is probabilistic, context-sensitive, and can change with model updates. That means you can't treat the model like a black-box SOP that always produces the same correction plan. You can, however, treat the model as "controlled assistance" — a source of suggestions that are subject to human review and traceability. But treating it that way still requires validation artifacts.
