Young People Turn to ChatGPT for Therapy, But Privacy Fails

The Rise of AI in Mental Health and the Privacy Dilemma

Artificial intelligence has become a significant tool in mental health support, especially for individuals who face barriers to traditional therapy. Unlike conventional therapy notes, which are protected by legal confidentiality, transcripts from interactions with AI chatbots lack such protections. This shift raises critical concerns about privacy, particularly for vulnerable groups like survivors of sexual assault.

In May 2023, Magistrate Judge Hon. Ona T. Wang issued an order requiring OpenAI to retain all ChatGPT user queries as part of a copyright infringement case involving The New York Times Company and Microsoft Corporation. While OpenAI appealed the decision, stating that it conflicts with their privacy commitments, the ruling remains in effect. With ChatGPT processing approximately 300 million new queries weekly, this decision has far-reaching implications.

AI as a Support System

For many individuals, especially those without access to traditional mental health services, AI tools offer a vital alternative. The Harvard Business Review highlighted that one of the leading uses of AI in 2025 is therapy and companionship. This trend reflects broader challenges in the U.S. healthcare system, where affordability and accessibility of mental health care have declined. A 2022 survey found that 90% of Americans believe there is a mental health crisis, with cost being a primary barrier to accessing care.

Young adults, in particular, are turning to AI apps over traditional verbal conversations. For survivors of trauma, these anonymous tools can provide a safe space to begin processing their experiences. Research indicates that timely access to quality care can reduce the long-term impact of trauma. AI tools, with their consistent and affirming responses, can help survivors feel supported and empowered to seek further assistance.

The Dual Nature of AI

While AI offers potential benefits, it also poses risks. The American Psychological Association has raised concerns about the readiness of AI to diagnose or support mental health disorders. Although some AI tools have received FDA clearance, none have met the agency's standards for clinical care. Despite these limitations, the APA acknowledges the potential for AI to play a supportive role in addressing the mental health crisis.

However, the current lack of safeguards leaves users vulnerable. AI transcripts, unlike traditional therapy notes, do not enjoy legal protection. This means that private conversations could be used as evidence in legal proceedings, potentially harming survivors who rely on these tools for support.

Legal Implications for Survivors

The risk of AI transcripts being weaponized is particularly concerning for sexual-assault survivors. A study published in the Journal of Interpersonal Violence highlights how survivors' narratives can be distorted in court. AI transcripts could exacerbate this issue, as early questions or fragmented disclosures might be misinterpreted as inconsistencies in a survivor’s account.

Survivors may also seek clarity on whether their experiences meet legal definitions of assault. However, such inquiries could be used against them in court, undermining their credibility. The absence of legal protections for AI interactions creates a dangerous precedent, potentially deterring survivors from seeking help altogether.

Protecting Privacy and Ensuring Safety

To ensure that AI supports rather than harms survivors, stronger privacy protections are essential. Platforms must implement clear warnings and anonymous modes that do not link to personally identifiable data. Educational institutions should also teach students how to use AI tools responsibly, helping them understand what information is safe to share and how to set boundaries.

Gen Z, millennials, and Gen Alpha, who have grown up with AI integrated into their lives, may be particularly susceptible to mistaking technological warmth for genuine security. As these groups turn to AI for mental health support, it is crucial to emphasize the importance of privacy and caution.

Conclusion

As AI continues to shape mental health care, it is imperative to address the ethical and legal challenges it presents. By prioritizing privacy and implementing robust safeguards, we can ensure that AI serves as a reliable and safe resource for those in need. Without these measures, the potential benefits of AI could be overshadowed by the risks it poses to vulnerable populations.