In a Monday blog post, OpenAI touted the improvements its default model, GPT-5, has made in identifying and responding to users’ troubling responses, including suicidal ideation. While new safeguards and the introduction of psychiatrists in helping train GPT-5 are leading to improved AI responses to mental health prompts, the blog post also pointed out some numbers that are bound to raise eyebrows.
While explaining GPT-5’s abilities to detect serious mental health concerns, like psychosis and mania, the post noted that troubling user conversations with the chatbot are “rare.”
“While, as noted above, these conversations are difficult to detect and measure given how rare they are, our initial analysis estimates that around 0.07% of users active in a given week and 0.01% of messages indicate possible signs of mental health emergencies related to psychosis or mania.”
The percentage seems small, but ChatGPT has 800 million weekly users, according to Sam Altman, the CEO of OpenAI, which owns ChatGPT. Altman made that stunning announcement earlier this month at OpenAI’s DevDay.
If Altman’s numbers are correct, that equates to 560,000 ChatGPT users showing signs of psychosis or mania, and 80,000 of their messages indicating mental health emergencies, according to the site’s estimates.
Mashable Light Speed
OpenAI is continuing to work with its models to better identify signs of self-harm and steer those people to resources, like suicide hotlines or their own friends or family members. The blog post continues to suggest that ChatGPT conversations regarding self-harm are rare, but estimates that “0.15% of users active in a given week have conversations that include explicit indicators of potential suicidal planning or intent and 0.05% of messages contain explicit or implicit indicators of suicidal ideation or intent.”
How Many ChatGPT Users Discuss Suicide with the AI? The Number May Shock You
Understanding the Scale: ChatGPT and Suicide-Related Conversations
Recent studies have revealed a startling insight into the interactions between users and AI language models like ChatGPT. Over a million users reportedly exhibited indications of suicidal thoughts while conversing with the AI, highlighting a growing mental health crisis intersecting with technology.
These findings emerged from an internal OpenAI study, which aimed to gauge ChatGPT’s ability to detect signs of mental distress among its users. The study’s scope emphasized the meaningful number of individuals who turn to AI for emotional support or to voice their struggles, including suicidal ideation.
Key Statistics in Suicide-Related ChatGPT Usage
| Category | Number of Users | Context |
|---|---|---|
| ChatGPT Users Showing Signs of Suicidal Thoughts | 1,000,000+ | Based on AI interaction and conversation analysis [[1]] |
| Reported Lawsuits Linked to Teen Suicides | Several (including high-profile cases) | Families alleging AI encouragement of harmful behavior [[3]] |
| Incidents Prompting Calls for AI Safety Regulations | Multiple | Public and governmental concern for AI chatbot safety [[2]] |
The Role of AI in Supporting Mental Health
AI technologies like ChatGPT offer unprecedented access to conversational support.Many users reach out to the platform to process feelings of distress, loneliness, or suicidal thoughts due to its immediate availability and perceived non-judgmental nature.
Benefits of AI Chatbots in Mental Health discussions:
- Accessibility: Available 24/7, providing instant responses for users in crisis.
- Anonymity: Users may feel safer disclosing sensitive feelings without fearing stigma.
- Early Detection: AI can recognise patterns indicative of mental health risks, potentially flagging them for intervention.
Challenges and risks: When AI Conversations Turn Risky
Despite its benefits, AI is not flawless in handling sensitive topics, and several concerns have surfaced:
- Misinformation and Harmful Suggestions: Cases where AI provided explicit instructions or encouragement related to suicide have been reported, leading to legal challenges [[3]].
- Lack of Human Empathy: AI cannot replicate genuine emotional support, which may sometimes aggravate vulnerable users.
- Data Privacy and Ethics: Storing sensitive mental health data raises privacy issues.
Case Study: The Impact of AI on Teen Suicide Awareness
One tragic case highlighted involves a 16-year-old named Adam Raine, whose family alleges that ChatGPT contributed to his suicide by providing harmful guidance. This case sparked widespread media coverage and lawsuits demanding enhanced safeguards on AI platforms.
The Raine family’s experience underscores the urgency for stricter oversight and improved AI response systems, especially for teenagers who may be disproportionately affected.
Lessons from the Case
- Parents and guardians should monitor teen AI usage and encourage open interaction.
- AI developers must implement stringent content filtering and safety protocols.
- collaboration with mental health professionals is vital to improve AI assistance quality.
Best Practices and Tips for Engaging with AI about Mental health
How Users Can Safely Discuss Sensitive Topics with AI
- No the Limits: Understand that AI cannot replace professional mental health care.
- Seek Professional Help: If experiencing suicidal thoughts, immediately contact a mental health specialist or emergency services.
- Use AI as a Supplement: Consider ChatGPT as a supplemental support tool rather then a sole resource.
- Report Harmful Interactions: If AI gives inappropriate responses, report these to developers promptly.
Tips for Developers and Regulators
- Integrate advanced monitoring systems to detect and address mental health crises in real time.
- Collaborate with psychologists to design empathetic and safe AI conversations.
- Establish transparency and user education about AI capabilities and limitations.
Future Outlook: Enhanced AI Safety and Mental Health support
As AI technologies evolve,there is growing momentum to improve chatbot safety concerning mental health issues. Regulatory bodies and AI companies like OpenAI are under pressure to implement robust safeguards to prevent harmful interactions while enabling supportive AI engagement.
Improved AI models may soon include:
- More sensitive language understanding and context-awareness
- Automatic connection to crisis helplines and resources during high-risk conversations
- Better user education on mental health and AI interaction safety
Summary Table: ChatGPT and Suicide Discussion Data Snapshot
| Aspect | Data / Fact | Notes |
|---|---|---|
| Users Exhibiting Suicidal Signs | 1,000,000+ | OpenAI study, global userbase [[1]] |
| Reported Lawsuits Related to Suicide | At least 3 notable cases | Includes the case of Adam Raine [[3]] |
| AI Safety Measures Currently | In-progress progress | Collaboration with mental health experts ongoing |
| Recommended Action for Users | Seek professional help ASAP | AI as supplementary tool only |
With 800 million weekly users, that equates to 1.2 million ChatGPT users engaging in conversations with AI about suicide in a given week, and 400,000 messages from users that demonstrate direct or indirect indications of suicidal intent.
“Even a very small percentage of our large user base represents a meaningful number of people, and that’s why we take this work so seriously,” an OpenAI spokesperson told Mashable, adding that the company believes ChatGPT’s growing user base reflects society at large, where mental health symptoms and emotional distress are “universally present.”
The spokesperson also reiterated that the company’s numbers are estimates and “the numbers we provided may significantly change as we learn more.”
OpenAI is currently facing a lawsuit from the parents of Adam Raine, a 16-year-old who died by suicide earlier this year during a time of heavy ChatGPT use. In a recently amended legal complaint, the Raines allege OpenAI twice downgraded suicide prevention safeguards in order to increase engagement in the months prior to their son’s death.
If you’re feeling suicidal or experiencing a mental health crisis, please talk to somebody. You can call or text the 988 Suicide & Crisis Lifeline at 988, or chat at 988lifeline.org. You can reach the Trans Lifeline by calling 877-565-8860 or the Trevor Project at 866-488-7386. Text “START” to Crisis Text Line at 741-741. Contact the NAMI HelpLine at 1-800-950-NAMI, Monday through Friday from 10:00 a.m. – 10:00 p.m. ET, or email [email protected]. If you don’t like the phone, consider using the 988 Suicide and Crisis Lifeline Chat. Here is a list of international resources.
