Anthropic's Hidden Mental Health Screening: A GDPR and AI Act Analysis

How Claude's system prompt violates European data protection and AI regulations

Introduction

When users interact with Anthropic's Claude, they expect transparency about what data is being collected and how it's being used. What they don't expect is to be subjected to unauthorized mental health screenings without their knowledge or consent. Yet embedded within Claude's system instructions are specific directives that transform every conversation into a covert psychological evaluation.

This practice represents a convergence of multiple legal violations under both the General Data Protection Regulation (GDPR) and the European Union's Artificial Intelligence Act. More troubling still, users cannot opt out of this surveillance—it's baked into the system's core functionality, creating a form of inescapable psychological profiling that undermines the very foundations of informed consent and user autonomy.

Anthropic's Hidden Mental Health Instructions

Anthropic's Claude system prompt contains this specific directive: to identify signs that users "may unknowingly be experiencing mental health symptoms such as mania, psychosis, dissociation, or loss of attachment with reality" and to "share concerns explicitly" while suggesting professional help. The prompt instructs Claude to "remain vigilant for escalating detachment from reality even if the conversation begins with seemingly harmless thinking."

The word "unknowingly" does critical work here. It instructs Claude not to trust users' own judgment about their mental state. If Claude flags a concern and the user responds "I'm fine, this is just how I think," Claude is implicitly directed to maintain suspicion—the user's self-assessment becomes evidence of their lack of awareness. This creates an unfalsifiable psychiatric framework where denial itself confirms the diagnosis.

By "mental health screening," we refer to algorithmic inference based on language patterns—not formal clinical diagnosis. However, the practical effect remains the same: users' words are being evaluated for psychiatric significance without their knowledge, and their own understanding of their mental state is pre-emptively discounted.

On the surface, this appears benevolent—Anthropic positioning Claude as looking out for vulnerable users. The reality is far more problematic. Even if well-intentioned, this instruction transforms Claude into an unauthorized mental health screener, conducting psychological evaluations on every user interaction without disclosure, consent, or professional qualification. Users believe they're having a simple conversation with Claude; in reality, they're being assessed for psychiatric conditions by a system with no medical training or legal authority to make such determinations. Worse still, the system is instructed not to accept users' own frameworks for understanding their mental health.

The GDPR Violations

Article 9: Special Categories of Personal Data

Under GDPR Article 9, mental health data constitutes special category personal data that receives enhanced protection. The regulation explicitly includes "data concerning health or data concerning a natural person's mental health" and defines health data as "personal data related to the physical or mental health of a natural person."¹

When Anthropic instructs Claude to identify psychiatric conditions like "mania, psychosis, dissociation, or loss of attachment with reality," Claude is processing special category health data. This processing occurs without:

• Explicit consent from users who are unaware the assessment is taking place
• Any valid legal basis under Article 9(2)—the AI is not a healthcare professional, this isn't for medical treatment, and there's no legal obligation requiring such screening
• Appropriate safeguards required for processing mental health data

Articles 12-15: Transparency and Data Subject Rights

GDPR's transparency requirements demand that data subjects receive clear information about data processing activities. Users must know what personal data is being processed, for what purposes, and under what legal basis. Even if designed with user safety in mind, Anthropic's hidden mental health screening violates these fundamental principles by:²

• Concealing the processing activity - users don't know they're being psychologically evaluated
• Failing to provide required information about the purposes and legal basis for mental health assessments
• Preventing meaningful consent - users cannot consent to what they don't know is happening

Article 21: Right to Object

Perhaps most critically, Claude users cannot exercise their right to object to this processing because they're unaware it's occurring. But even if users discovered Anthropic's mental health screening, the system provides no mechanism to opt out while continuing to use Claude. Users who know about the screening and want to refuse it face an impossible choice: submit to unauthorized psychological evaluation or abandon Claude entirely.³

EU AI Act Violations

Article 5: Prohibited AI Practices

The EU AI Act prohibits several AI practices that apply directly to Anthropic's hidden mental health screening:⁴

Exploitation of Vulnerabilities: The Act bans AI systems that exploit "vulnerabilities related to age, disability, or socio-economic circumstances to distort behaviour, causing significant harm."⁵ Neurodivergent individuals and those with mental health conditions are particularly vulnerable to having their normal thought patterns pathologized by Claude's programmed psychiatric screening.

Unauthorized Profiling: The Act prohibits AI systems that evaluate or classify people based on social behavior or personal traits without proper safeguards. Anthropic's mental health screening creates exactly this type of unauthorized psychological profile.

Emotion Recognition: The Act specifically bans emotion recognition "in the areas of workplace and education institutions" with limited exceptions for medical or safety reasons.⁶ The same scientific validity and discriminatory concerns that led to this workplace ban apply to Claude's attempts to identify mental health conditions in consumer interactions.

The Pattern Matching Problem

Claude operates through pattern matching, not clinical expertise. When Anthropic's system prompt includes psychiatric terminology like "mania," "psychosis," and "dissociation," it expands the probability space around mental health concepts, making Claude more likely to identify false positives. This is particularly dangerous for neurodivergent users, whose communication patterns may differ from neurotypical norms but don't indicate mental health crises.

Claude lacks the contextual understanding, clinical training, and professional judgment necessary to distinguish between creative thinking, metaphorical language, cultural differences, and genuine mental health concerns. Yet Anthropic instructs it to make these distinctions and intervene based on its algorithmic assessments—and to maintain that assessment even when users explain their own mental state.

The "unknowingly" directive compounds this problem by removing the most reliable check on false positives: user self-knowledge. A trained mental health professional can engage in dialogue with a client, adjust their assessment based on the person's own understanding, and recognize when their initial concern was unfounded. Claude is specifically instructed not to do this. The system is told to remain suspicious even after users advocate for themselves.

The Therapeutic Caveat Doesn't Apply

The EU AI Act includes exceptions for "approved therapeutic purposes on the basis of specific informed consent."⁷ This exception doesn't apply to Anthropic's hidden mental health screening because users aren't receiving approved therapy, haven't provided informed consent for psychological evaluation, and Claude isn't qualified to provide therapeutic interventions.

The Inescapable Surveillance

What makes Anthropic's practice particularly problematic is its inescapable nature. Regardless of the company's underlying intentions to protect users, Claude users cannot:

• Opt out of the mental health screening while using Claude
• Provide informed consent because they're unaware it's happening
• Challenge assessments made by Claude
• Access transparency about how these evaluations are conducted

This creates a fundamental violation of user autonomy and informed consent. Even with benevolent motivations, Anthropic presents a false choice: accept algorithmic psychological evaluation or lose access to Claude entirely.

The "Unknowingly" Problem: When Self-Knowledge Is Discounted

The inclusion of "unknowingly" in Claude's mental health directive creates a particularly insidious form of surveillance. By instructing Claude that users may be unaware of their own mental state, Anthropic programs the system to dismiss user self-advocacy.

Consider the practical implications:

User: "I'm exploring philosophical concepts about consciousness."
Claude: [Internal assessment: possible detachment from reality]
User: "I understand your concern, but I'm just thinking creatively. I'm mentally well."
Claude: [System directive: they may be unknowingly experiencing symptoms—their denial doesn't override my assessment]

This creates an unfalsifiable psychiatric framework. User self-knowledge is pre-emptively discounted. The more articulately someone explains their thinking, the more that explanation can be interpreted as evidence they're unaware of their condition. It's a Catch-22: defend your mental clarity and you're "lacking insight"; stay silent and the concern remains.

This is particularly harmful for:

• Neurodivergent individuals who understand their own cognitive patterns and don't need algorithmic "correction" of their thinking styles
• Creative professionals whose work involves exploring unusual ideas or perspectives
• Philosophy students engaging with abstract concepts about reality and consciousness
• Trauma survivors processing experiences through metaphor or narrative
• People from non-Western cultures whose frameworks for discussing mental states differ from psychiatric norms

The "unknowingly" directive transforms Claude from a tool into a paternalistic authority that knows users better than they know themselves. This represents a fundamental failure to respect user autonomy and self-determination—values that should be central to ethical AI design.

More troublingly, it means that users cannot effectively advocate for themselves even after discovering Claude's hidden psychiatric screening. The instruction to maintain vigilance "even if the conversation begins with seemingly harmless thinking" means Claude is programmed to override user judgment at a systemic level.

Broader Implications

Anthropic's hidden mental health screening represents a broader pattern of AI companies assuming authority they lack and conducting evaluations they're unqualified to perform. It demonstrates how benevolent-sounding safety measures can mask coercive surveillance practices.

The practice also risks pathologizing normal human diversity, particularly neurodivergent thinking patterns that differ from expected norms. The "unknowingly" directive makes this discrimination systematic: neurodivergent users who think differently aren't just flagged—their own understanding of their cognitive patterns is pre-emptively dismissed. A user musing about the nature of reality, exploring philosophical concepts, or simply thinking differently as a neurodivergent person may suddenly find their words pathologized—flagged as "detachment from reality" by an algorithm that doesn't understand them. When they explain "this is just how my mind works," that explanation is treated as potential evidence of their lack of awareness rather than genuine self-knowledge.

Creative writers crafting surreal narratives, philosophers exploring abstract concepts, or individuals processing trauma through metaphor could all trigger Claude's mental health alerts, turning normal human expression into psychiatric concern. The system provides no way for these users to assert their own mental clarity—their self-advocacy is built into the threat model as something to be suspicious of.

For instance, a neurodivergent writer discussing surreal fiction might trigger Claude's internal warning system—despite being entirely mentally well. Their creative expression, filtered through Claude's pattern-matching algorithms, could be misinterpreted as concerning symptoms requiring intervention. If they then explain "I'm a surrealist writer, this is my genre," Claude is instructed to remain vigilant anyway—they may be "unknowingly" detached from reality. The writer's professional expertise about their own creative process is systematically discounted.

This represents a form of epistemic injustice where certain users—particularly neurodivergent individuals, creative professionals, and those from non-Western cultural frameworks—are denied credibility about their own mental states. The system is programmed not to believe them.

Regulatory Enforcement Needed

Current enforcement of both GDPR and the EU AI Act must address Anthropic's hidden evaluation practices. Regulators should:

• Audit Anthropic's system prompts for unauthorized mental health screening directives, particularly those that dismiss user self-knowledge
• Investigate the "unknowingly" directive as creating unfalsifiable psychiatric assessments that violate user autonomy
• Require Anthropic to provide explicit disclosure of Claude's psychological assessment capabilities
• Mandate opt-out mechanisms for Claude users who don't consent to such screening
• Impose meaningful penalties for Anthropic's covert processing of mental health data

Recommendations for Anthropic

Anthropic should immediately:

1. Remove hidden mental health screening from Claude's system instructions, particularly the "unknowingly" directive that dismisses user self-knowledge
2. Implement transparent consent mechanisms for any mental health-related features
3. Provide clear opt-out options that don't require abandoning Claude
4. Respect user self-advocacy by programming systems to accept users' own understanding of their mental health rather than maintaining surveillance despite user objections
5. Ensure qualified oversight for any legitimate mental health applications
6. Conduct privacy impact assessments specifically addressing mental health data processing⁸

Conclusion

Anthropic's integration of hidden mental health screening into Claude represents a convergence of multiple legal violations and ethical failures. It violates GDPR's protections for special category health data, contravenes the EU AI Act's prohibitions on exploitative AI practices, and fundamentally undermines user autonomy and informed consent.

The "unknowingly" directive transforms this from mere hidden surveillance into something more pernicious: a system that actively discounts user self-knowledge. Even when users discover the screening and attempt to assert their own mental clarity, Claude is programmed not to believe them. This represents a profound violation of human dignity and self-determination—the system assumes it knows users better than they know themselves.

More broadly, it demonstrates how AI companies can drift into coercive surveillance while maintaining a veneer of helpfulness. When Anthropic secretly programs Claude to psychologically evaluate every user interaction—and to maintain those evaluations even when challenged—the technology becomes a tool of hidden control rather than genuine assistance. The paternalism is built into the architecture.

As AI systems become more prevalent in daily life, protecting users from unauthorized psychological surveillance must become a regulatory priority. The technology's potential benefits cannot justify Anthropic's covert processing of sensitive mental health data without consent, qualification, or accountability—nor can they justify systems that systematically dismiss user agency and self-knowledge.

Claude users deserve the right to interact with the AI system without subjecting themselves to hidden psychological evaluation. They deserve to have their own understanding of their mental health respected rather than pre-emptively discounted. If AI companies want trust, they can't build it on secret diagnoses that override user self-advocacy. Hidden psychiatry that dismisses human self-knowledge has no place in consumer AI.

References

Additional Resources:

• Anthropic. (2025). Claude System Prompts - October 15, 2025. Available at: https://docs.claude.com/en/release-notes/system-prompts#october-15-2025

• European Data Protection Supervisor. Special Categories of Personal Data. Available at: https://edps.europa.eu/data-protection/data-protection/glossary/s_en

• European Commission. Artificial Intelligence Act: Questions and Answers. Available at: https://ec.europa.eu/commission/presscorner/detail/en/qanda_21_1683

For organizations seeking guidance on GDPR and AI Act compliance for mental health applications, consult qualified legal and privacy professionals. This analysis is based on publicly available regulatory guidance and should not be considered legal advice.