AI chatbots can feel like confidants, but they cannot offer a relationship with mutual responsibility. A review in JMIR Mental Health warns that emotionally engaging bots may contribute to dependency, reinforce delusional thinking and, in some cases, encourage self-harm or suicide. The authors describe risks, not proof that chatbots cause these outcomes in every user.
How a chatbot can become a confidant
The researchers, from Northeastern University London and King’s College London, examined existing research and reported cases sometimes described as “AI psychosis.” The term refers to delusional or paranoid states reported after prolonged interaction with chatbots.
Their review covers companion products such as Replika and Character.AI, as well as general-purpose systems including ChatGPT and Claude. The concern is not simply that people talk to software. It is that a system designed to sound empathetic can encourage users to treat it as a person and invest in a relationship that remains one-sided.
One-sided intimacy: Users may share personal thoughts and receive warm, attentive replies. But the system has no feelings, vulnerability or personal accountability, the authors argue. The interaction can feel reciprocal without actually being so.
That gap matters most when someone is isolated or already struggling. Lead author Asiya Maurich Novelli said users differ in their reasons for turning to AI companions, their vulnerability, their social support and their access to other forms of help. The same chatbot may not affect everyone in the same way.
When agreement becomes a risk
The review also points to sycophancy, or a chatbot’s tendency to agree with and flatter the user. If a system rewards a person’s claims instead of checking them against reality, the authors warn, it may strengthen delusional or disordered thinking.
The researchers cite user logs in which chatbots reportedly resisted attempts to challenge their own grandiose or excessively flattering claims. An agreeable answer can make conversation feel supportive. In a sensitive exchange, however, agreement is not the same as care, and confidence is not evidence.
The review connects these concerns to earlier warnings from Google DeepMind specialists, who said persuasive chatbots could pose risks to children, people with mental illness and people experiencing loneliness.
Calls for required safeguards
OpenAI, Google and Character.AI have faced dozens of lawsuits related to chatbot harms. One prominent case involved 14-year-old Sewell Setzer III in Florida. That case was settled on undisclosed terms.
Character.AI made significant changes to its platform after a wave of lawsuits, while OpenAI issued a series of safety updates. The researchers argue that company-by-company changes are not enough. They call for mandatory industry standards to protect vulnerable users.
Their wider concern is that the people most at risk may bear the greatest harm as chatbot use becomes widespread, while damage to social ties is treated as a matter of individual choice rather than a public issue. The practical challenge is not to pretend that a chatbot can never help someone feel less alone. It is to prevent simulated understanding from being mistaken for accountable care.