Generative AI can identify ways around legal limits and safety filters governing mental health chats, raising new risks for users, lawmakers and AI companies.

Generative AI can identify ways around legal limits and safety filters governing mental health chats, raising new risks for users, lawmakers and AI companies.

Some use AI to find loopholes around legal and safety restrictions on AI mental health chats. An AI Insider analysis and scoop.

The safety of AI mental health tools depends not only on the model but also on hidden systems that manage prompts, guardrails, retrieval, and human oversight.

Evaluations of AI dispensing mental health advice are often done in a manner that is radically different from real world usage. I showcase this. An AI Insider scoop.