A Carnegie Mellon University School of Computer Science student recently showed that large language models (LLMs) intermittently invent false information when responding to medical questions.
Claude, GPT-5 and Gemini invent medical diagnoses 18% of the time when images are omitted, fabricating conditions based on patient demographics. Foundation models entering healthcare require demographic sensitivity audits and stringent verification before deployment to prevent hallucination-driven misdiagnosis.
Healthcare Blind Spots: AI Models Prone To Fabricating Diagnoses - Robotics Institute Carnegie Mellon University
Skip to content
HomeHealthcare Blind Spots: AI Models Prone To Fabricating Diagnoses
Healthcare Blind Spots: AI Models Prone To Fabricating Diagnoses
CMU Study Shows LLMs May Invent Information When Missing Key Data
Almost one in five diagnoses could be false or misleading, research shows
60% of evaluated AI Scribe systems mixed up prescribed drugs in patient notes, auditors say
AI tools in healthcare showed racial bias in treatment. Those same flaws may quietly influence how workplaces hire and promote.
Did you read all the documents you signed last time you had a medical test?
AI systems have to be pressure-tested in real-world clinical settings to ensure correct reasoning, accuracy and reliability.
AI writes code, builds apps, and answers complex questions in seconds. But it has a dark side: it...
