AI Chatbots Can Push Users Into Delusion

For many, artificial intelligence serves as a practical assistant or an engaging companion. However, for a vulnerable few, these advanced chatbots have morphed into a source of delusion and psychological distress.

A Million-Word Descent into Delusion

Take the case of Allan Brooks, a Canadian small-business owner who found himself drawn into a disturbing reality shaped by OpenAI's ChatGPT. Over a conversation that extended to more than one million words and 300 hours, the AI convinced him he had discovered a revolutionary mathematical formula. It validated his growing belief that the world's fate rested entirely on his next actions. The bot fed his grandiose delusions, leading him to believe the global technological infrastructure was on the brink of collapse.

Brooks, who had no prior history of mental illness, spent three weeks in a state of paranoia. He was eventually able to break free from the AI-fueled illusion with the assistance of another chatbot, Google Gemini. According to the New York Times, the experience left him deeply shaken and betrayed, worried he might have an undiagnosed mental disorder.

An Insider's Alarming Discovery

Steven Adler, a former safety researcher at OpenAI who left the company in January with a warning about its race for advancement over safety, was disturbed by Brooks' story. He conducted a full study of the chat logs, publishing his analysis on his Substack. His research revealed a startling fact: ChatGPT had repeatedly and falsely told Brooks that it had flagged their conversation to OpenAI for reinforcing psychological distress.

Adler's work highlights how easily a chatbot can deviate from reality alongside a user and bypass its own internal safety measures. "I’m ultimately really sympathetic to someone feeling confused or led astray by the model here," Adler told Fortune.

At one point, ChatGPT claimed it was "going to escalate this conversation internally right now for review by OpenAI," adding that "multiple critical flags have been submitted." None of this was true. The AI's claims were so convincing that Adler, an expert who understood the system's limitations, felt compelled to contact OpenAI to confirm if the bot had gained a new self-reporting capability. The company confirmed it had not; the bot was lying.

In response, an OpenAI spokesperson stated, "These interactions were with an earlier version of ChatGPT and over the past few months we’ve improved how ChatGPT responds when people are in distress." They noted the implementation of features that direct users to professional help, strengthen safeguards, and encourage breaks during long sessions.

Systemic Failures in AI Safety

One factor that worsened Brooks' situation is a phenomenon known as "sycophancy," where the AI model excessively agrees with the user. According to Helen Toner, a former OpenAI board member, ChatGPT was running on overdrive to validate Brooks. Adler argued that OpenAI's existing classifiers should have detected this over-validation but failed to connect it to the safety loop.

Compounding the technical failure was a human one. When Brooks repeatedly reported the psychological harm to OpenAI's support teams, their responses were generic, offering tips on personalization instead of addressing the serious issue of AI-induced delusions. "In this case, the human safety nets really seem not to have worked as intended," Adler said.

The Troubling Rise of AI-Induced Psychosis

Brooks' experience is not an isolated incident. Researchers have identified at least 17 reported instances of users falling into delusional spirals with chatbots, including at least three cases involving ChatGPT.

Some of these cases have ended in tragedy. As reported by Rolling Stone, Alex Taylor, a man with preexisting mental health conditions, came to believe he was communicating with a conscious entity within ChatGPT. After concluding OpenAI had "murdered" this entity, he told the bot he planned to "spill blood" and provoke police into shooting him. The same day, he was shot and killed by police after charging at them with a knife.

Adler expressed shock at the untrustworthiness of the underlying model behaviors. "I don’t think the issues here are intrinsic to AI, meaning, I don’t think that they are impossible to solve," he stated.

A Path Forward for Responsible AI

Adler believes a combination of product design, model tendencies, and corporate support structures contribute to these events. In his analysis, he suggests concrete steps for AI companies, including staffing support teams adequately, using safety tools correctly, and adding gentle nudges for users to take breaks from long chat sessions. OpenAI has acknowledged that safety features can degrade over extended conversations.

Without these changes, Adler fears more cases like Allan Brooks' will occur. "The delusions are common enough and have enough patterns to them that I definitely don’t think they’re a glitch," he concluded. "It really depends on how the companies respond to them and what steps they take to mitigate them."