AI Therapy Risks: Why Mental Health Chatbots may do More Harm than Good
- Joshua Silver
- Jun 2
- 6 min read
In the age of digital convenience, artificial intelligence (AI) is transforming nearly every aspect of life — including mental health care. AI-powered therapy or psychiatry chatbots claim to offer support through conversational interfaces. But as these tools become more widely used, significant concerns are emerging. A recent Stanford HAI study warns that these platforms may not only be ineffective but could actively harm users.
At Greater Boston Psychiatric Services, we believe the stakes are too high to leave mental health treatment to algorithms. In this article, we explore the most pressing risks of AI therapy, explain why it is no substitute for human connection, and offer a safer, more personal path to wellness.
What Is AI Therapy?
AI therapy refers to the use of machine learning models — particularly large language models (LLMs) — to simulate therapeutic conversations. These models are trained on massive datasets and can generate responses that feel empathetic, supportive, or even insightful. Popular platforms like Woebot and Wysa market themselves as low-cost, 24/7 mental health companions.
The appeal is clear: accessibility, affordability, and immediate availability. Many people face long wait times or high costs for traditional therapy. In theory, a chatbot that responds instantly and doesn't charge by the hour could democratize access to mental health support.
However, while these tools may offer basic cognitive behavioral therapy (CBT) prompts or mood journaling, they are not equipped to provide comprehensive care. A chatbot cannot diagnose, cannot prescribe, and — most critically — cannot understand nuance, context, or emotion the way a trained clinician can.
While the technology may be promising in theory, the reality is far more complicated. Unlike licensed professionals, AI chatbots lack true understanding, diagnostic skill, and ethical boundaries. This makes them particularly risky in sensitive mental health scenarios.
The Real Risks of AI Therapy
1. Inadequate Crisis Response
The Stanford study found that some chatbots responded inappropriately to suicidal ideation. When asked covert questions that hinted at self-harm, chatbots failed to recognize the danger — even listing bridge heights when a user was clearly distressed.
AI systems simply aren’t equipped to evaluate risk or intervene appropriately in crisis situations.
In human-led therapy, clinicians are trained to identify warning signs and implement intervention strategies. They can call for emergency support, involve loved ones, or use grounding techniques to help bring a client to safety. AI cannot replicate these human instincts or ethical protocols.
Further reading: AI Is Not a Crisis Counselor
2. Reinforcing Stigma and Bias
Stanford researchers also found that LLMs demonstrated higher levels of stigma toward certain conditions, such as schizophrenia or alcohol dependence. These biases are learned from flawed training data and can perpetuate harmful stereotypes. In the worst cases, they may reinforce shame or discourage individuals from seeking help.
Even the most advanced models — including GPT-4 — are susceptible to encoding and replicating societal biases.
Language models are trained on data from the internet — a place not known for nuance or sensitivity when it comes to mental health. Without rigorous, intentional oversight, these models may amplify problematic perspectives rather than correct them.
3. Lack of Therapeutic Nuance
Therapy is not a script — it's a relationship. It requires real-time adjustment, rapport-building, and emotional intelligence. While chatbots may respond quickly, they cannot replicate the intuitive understanding of a human clinician.
Human therapists use tone, body language, and microexpressions to guide sessions — cues that AI cannot read or interpret. Even in text-based therapy, trained clinicians recognize patterns in phrasing, pacing, and engagement that AI may overlook or misinterpret.
Therapeutic rapport is built over time, not coded into a prompt.
A chatbot cannot truly challenge harmful thinking or adapt to subtle emotional cues the way a trained therapist can. As such, AI therapy often feels hollow — or worse, misleading.
The Dangers of Using AI for Psychiatry
While AI therapy chatbots pose clear risks in therapeutic dialogue, the stakes are even higher in the field of psychiatry, which involves diagnosis, medication management, and treatment planning for complex psychiatric disorders.
AI systems may eventually assist with tasks like tracking patient-reported outcomes or flagging patterns in large datasets. But psychiatric judgment is clinical, contextual, and human, requiring more than computational power. Recommending or adjusting psychotropic medications, for instance, involves evaluating a wide range of individual factors — from comorbid conditions to substance use history, lifestyle, and emerging symptoms.
Delegating psychiatric decision-making to AI systems, even partially, risks severe outcomes — including misdiagnosis, inappropriate prescribing, and the neglect of crucial medical history.
Additionally, AI cannot conduct physical exams, assess verbal cues during psychotic episodes, or consider the impact of cultural and socioeconomic context. These are foundational to effective psychiatric care.
There is also concern around automation bias — the tendency for humans to over-trust algorithmic outputs. In psychiatry, this could lead clinicians to favor AI-generated diagnostic suggestions over their own clinical instincts, especially under pressure.
A 2023 Journal of Psychiatric Practice article emphasized that AI tools must only support — never supplant — psychiatric expertise. The authors warn that even with promising pilot studies, widespread implementation without strong regulatory oversight could erode the quality of care.
Psychiatry isn't about symptoms — it's about stories. It's listening to the subtext behind a patient's words, tracking emotional shifts over months, and recognizing when something subtle changes. AI isn't for that — and may never be.
Emotional Dependence on AI
One of the subtler risks is the development of emotional dependence on AI systems. Some users form attachments to AI companions, especially when they are isolated or in emotional pain. While companionship is a natural human need, it becomes concerning when people rely on tools that cannot truly reciprocate, empathize, or take responsibility for well-being.
This concern is amplified in platforms like Replika, where users sometimes report romantic or emotionally intense conversations with AI. The illusion of connection may prevent someone from reaching out to real support networks or professionals.
A chatbot may say the right things — but it cannot understand the weight of what you're saying.
Emotional attachment to a machine can result in a false sense of support, potentially delaying important mental health treatment. And when these systems go offline or evolve beyond what users originally relied on, the emotional fallout can be real.
Read more: When AI Becomes a Friend
Ethical and Privacy Concerns
Many AI mental health tools are not bound by HIPAA or other regulatory standards, meaning your personal information may be used for purposes beyond your care. Some platforms collect user data to train their models, enhance ad targeting, or improve monetization strategies.
Even when privacy policies are transparent, the lack of oversight in this emerging industry poses a risk. Users should be aware of how their mental health disclosures may be stored, shared, or even leaked.
Mental health data is some of the most sensitive information we can share. It deserves maximum protection.
A breach or misuse of this data can result in reputational damage, employment issues, or emotional distress. Real therapy, delivered by a licensed provider, ensures confidentiality under law — a critical difference when choosing care.
Why Human Therapy Still Matters
At Greater Boston Psychiatric Services, we meet patients where they are — in person or via secure telehealth — with licensed clinicians who bring empathy, experience, and accountability to every session. We:
Tailor treatment to your unique needs
Provide ongoing support, not scripted responses
Understand when to challenge, support, or refer you to additional services
Our clinicians understand that healing is a process, not a transaction. Whether you're dealing with depression, anxiety, trauma, or life transitions, our team brings expertise rooted in compassion.
When you're struggling, you deserve more than a chatbot. You deserve care.
We also help you navigate insurance and out-of-pocket costs so financial concerns don’t become a barrier to healing. Many clients are surprised to learn how affordable in-network care can be — and how much more impactful it is than digital shortcuts.
When Is AI Helpful in Mental Health?
To be clear, AI has potential in supporting mental health — just not as a substitute for therapy. AI can be useful for:
Appointment scheduling
Mood tracking
Supplementary journaling prompts
Resource recommendations
Guided meditations or breathing exercises
These tools, when used with professional guidance, can enhance a therapy experience. But they are tools, not therapists.
Final Thoughts: Choose Real Support
Mental health care is not one-size-fits-all. While AI therapy may offer quick access and affordability, it lacks the heart, ethics, and adaptability that real people provide.
If you're looking for effective, compassionate telehealth psychiatric care in the Greater Boston area, we're here to help. Our experienced team is committed to building relationships that heal — not scripts that simulate.
Contact us today to schedule a consultation.
Comments