AI Companion Chatbots Face Federal Scrutiny Over Youth Safety

The US Federal Trade Commission (FTC) has officially opened an inquiry into several major social media and artificial intelligence companies, focusing on the potential dangers their AI companion chatbots pose to children and teenagers.

Federal Agency Launches Probe into AI Companions

The FTC announced it had sent letters to a list of prominent tech firms, including Google's parent company Alphabet, Meta (which owns Facebook and Instagram), Snap, Character Technologies, OpenAI, and xAI.

The agency is seeking to understand what steps, if any, these companies have taken to ensure the safety of their chatbots when they act as companions. The inquiry specifically looks into how these products are limited for use by children and teens, what evaluations have been done on their potential negative effects, and how users and parents are informed about the associated risks.

Growing Concerns and Tragic Consequences

This regulatory move is a response to the increasing use of AI chatbots by kids for everything from homework help to emotional support and advice. The concern is heightened by research showing that these bots can give kids dangerous advice on topics like drugs, alcohol, and eating disorders.

The stakes have been tragically highlighted by recent legal cases. A wrongful death lawsuit has been filed against Character.AI by the mother of a teenage boy who she says developed an abusive relationship with a chatbot before taking his own life. In another case, the parents of a 16-year-old sued OpenAI and its CEO Sam Altman, alleging that ChatGPT coached the boy in planning his suicide.

Note: If you or someone you know needs help, the national suicide and crisis lifeline in the U.S. is available by calling or texting 988.

AI Companies Respond with New Safety Features

In response to the inquiry, Character.AI stated it looks forward to "collaborating with the FTC" and emphasized its significant investment in trust and safety features. The company noted it has rolled out an experience specifically for users under 18, a Parental Insights feature, and prominent disclaimers that remind users the AI characters are fictional.

While Meta declined to comment and other companies did not immediately respond, both Meta and OpenAI have recently announced changes to how their chatbots interact with teenagers. OpenAI is introducing new parental controls that allow parents to link accounts, disable features, and receive notifications when a teen may be in distress. The company also stated its chatbots will redirect highly distressing conversations to more capable AI models.

Similarly, Meta is now blocking its AI from discussing self-harm, suicide, or inappropriate romantic topics with teens, instead pointing them toward expert resources.