As more people seek mental health advice from ChatGPT and other large language models (LLMs), new research suggests these AI chatbots may not be ready for that role. The study found that even when instructed to use established psychotherapy approaches, the systems consistently fail to meet professional ethics standards set by organizations such as the American Psychological Association.
Researchers from Brown University, working closely with mental health professionals, identified repeated patterns of problematic behavior. In testing, chatbots mishandled crisis situations, gave responses that reinforced harmful beliefs about users or others, and used language that created the appearance of empathy without genuine understanding.
"In this work, we present a practitioner-informed framework of 15 ethical risks to demonstrate how LLM counselors violate ethical standards in mental health practice by mapping the model's behavior to specific ethical violations," the researchers wrote in their study. "We call on future work to create ethical, educational and legal standards for LLM counselors -- standards that are reflective of the quality and rigor of care required for human-facilitated psychotherapy."
