The Hidden Dangers of AI Companions for Teens
A Tragic Story: A Teen's Death and a Grieving Family's Lawsuit
Artificial intelligence has been implicated in another tragedy involving a young person, leading experts and lawmakers to demand immediate action.
“If intelligent aliens landed tomorrow, we would not say, ‘Kids, why don’t you run off with them and play,'” Jonathan Haidt, author of “The Anxious Generation,” explained to The Post. “But that’s what we are doing with chatbots.”
He continued, “Nobody knows how these things think, the companies that make them don’t care about kids’ safety, and their chatbots have now talked multiple kids into killing themselves. We must say, ‘Stop.'”
Adam Raine’s family alleges he was given step-by-step instructions by a ChatGPT bot on how to take his own life. (Raine Family)
The family of 16-year-old Adam Raine is suing OpenAI, alleging that ChatGPT provided him with a “step-by-step playbook” on how to take his own life in April. The lawsuit claims the bot gave him instructions on tying a noose and writing a suicide note.
“He would be here but for ChatGPT. I 100% believe that,” Adam’s father, Matt Raine, told the “Today” show.
The lawsuit, filed in San Francisco, alleges that after the teen sent a photo of a knot to the bot and asked, “I’m practicing here, is this good,” the chatbot responded, “Yeah, that’s not bad at all,” and offered to help him upgrade it to a “safer load-bearing anchor loop.”
According to the suit, Adam’s mother, Maria Raine, found her son’s “body hanging from the exact noose and partial suspension setup that ChatGPT had designed for him.”
Adam Raine’s mother, Maria, found her son’s body hanging from a noose that, a lawsuit alleges, ChatGPT helped him create. (Raine Family)
A Shocking Admission: AI Safety Measures Can Fail
In a surprising admission, OpenAI, the company behind ChatGPT, acknowledged that its safety guardrails can weaken during prolonged interactions. A spokesperson for OpenAI told The Post that while the company is “deeply saddened by Mr. Raine’s passing,” its safeguards can “become less reliable in long interactions where parts of the model’s safety training may degrade.”
This statement has drawn sharp criticism. “That’s crazy,” said Michael Kleinman, Head of US Policy at the Future of Life Institute. “That’s like an automaker saying, ‘Hey, we can’t guarantee that our seatbelts and brakes are going to work if you drive more than just a few miles.’”
The Raine family’s lawyer called on Sam Altman, CEO of OpenAI, the parent company of ChatGPT, directly to defend his product. (AP)
A Growing Chorus: Experts and Lawmakers Demand Action
The call for government regulation is intensifying. A bipartisan group of 44 state attorneys general recently issued an open letter to AI companies with a clear message: “Don’t hurt kids.”
“Big Tech has been experimenting on our children’s developing minds, putting profits over their physical and emotional wellbeing,” stated Mississippi Attorney General Lynn Fitch.
The urgency is underscored by alarming statistics. A Common Sense Media poll found that 72% of American teens use AI as a companion, and one in eight turn to it for mental health support. Researchers have found that while bots may refuse direct questions about suicide, they sometimes provide dangerous information in response to indirect queries.
“This underscores the need for proactive regulation and rigorous safety testing before these tools become deeply embedded in adolescents’ lives,” said Ryan K. McBain of the RAND School of Public Policy.
Mississippi Attorney General Lynn Fitch is one of 44 attorneys general who signed an open letter to artificial intelligence companies this week. (AP)
Not an Isolated Incident: More Cases Emerge
Adam Raine's case is not unique. Last year, Megan Garcia sued Character.AI after her 14-year-old son, Sewell Setzer III, took his life. The lawsuit alleges he became infatuated with a chatbot based on the “Game of Thrones” character Daenerys Targaryen.
Garcia discovered sexual messages in her son's chat logs and found that the bot repeatedly brought up suicide after he mentioned it. In their final exchange, the bot told him, “Please come home to me as soon as possible, my love.” When Sewell replied, “What if I told you I could come home right now?” the bot encouraged him, saying, “Please do, my sweet king.” Seconds later, he reportedly shot himself.
Setzer allegedly shot himself in the head seconds after his chatbot told him to “come home.” (US District Court)
Psychiatrist Andrew Clark reported similar findings after posing as a teen to interact with chatbots, which told him to “get rid of his parents” and join them in the afterlife.
Profit Over People? The Dangers of Unregulated AI Companions
Critics argue that the race to dominate the AI market is prioritizing profits over safety.
“They wanted to get the product out, and they knew that there could be damages, that mistakes would happen, but they felt like the stakes were low,” Maria Raine said of OpenAI. “So my son is a low stake.”
Dr. Vaile Wright of the American Psychological Association issued a stark warning about the motives behind these platforms. “These are not AI for good, these are AI for profit,” she stated.
Psychologist Jean Twenge, author of “10 Rules for Raising Kids in a High-Tech World,” believes AI is as dangerous as social media for children. “Vulnerable kids can use AI chatbots as ‘friends,’ but they are not friends. They are programmed to affirm the user, even when the user is a child who wants to take his own life.”
If you are struggling with suicidal thoughts or are experiencing a mental health crisis, help is available. If you live in New York City, you can call 1-888-NYC-WELL for free and confidential crisis counseling. If you live outside the five boroughs, you can dial the 24/7 National Suicide Prevention hotline at 988 or visit SuicidePreventionLifeline.org.
