You’d think that it would be obvious that it’s a terrible idea to ask an AI chatbot for a medical consultation, but hundreds of millions of people do it every week.
In one near-death example now the subject of a lawsuit, ChatGPT advised a pastor to ignore symptoms of a pulmonary embolism – a condition with a mortality rate of 30%. In fact, the advice it offered very likely made things worse …
AI chatbots can be useful in a wide variety of ways, but using them as a substitute for a consult with a doctor isn’t one of them. Even using them in place of a Google search is inadvisable given their track-record of confidently hallucinating.
Yet even OpenAI’s own records show that literally hundreds of millions of people ask ChatGPT health questions every week. Similarly, an analysis of searches on Microsoft’s Copilot found that health questions were one of the most commonly asked in the mobile app.
If you needed any evidence of the potential consequences of relying on a chatbot for medical advice, a lawsuit by a Florida pastor provides an illuminating example. The New York Times reports that he is suing OpenAI after ChatGPT told him he didn’t need to worry about symptoms of a pulmonary embolism.
