OpenAI Scraps Chat Sharing After Privacy Disaster

OpenAI has officially removed a feature from ChatGPT that allowed user conversations to become discoverable on search engines like Google. This move comes after an investigation revealed significant privacy breaches resulting from the tool.

Investigation Uncovers Major Data Leaks

The change was prompted by a probe from online research expert Henk van Ess of Digital Digging. His analysis of more than 500 publicly shared ChatGPT conversations uncovered what he described as a trove of “self-incrimination and leaked confidential data.” The findings painted a stark picture of the risks associated with making AI chats public.

The Alarming Nature of Exposed Conversations

Among the publicly indexed chats, the investigation found incredibly sensitive information. Corporate executives and government employees were found to have discussed and admitted to crimes, exposed insider trading schemes, and even laid out plans for cyberattacks targeting groups like Hamas.

The security vulnerabilities of the AI were also on full display. Henk van Ess reported that one user managed to find a workaround to coerce ChatGPT into creating “inappropriate content involving minors,” even after the system initially refused. “This demonstrated both the platform's vulnerabilities and the concerning intentions some users bring to AI interactions,” he wrote.

In another alarming example, a supposed health worker asked the AI to roleplay as an oncologist for an elderly male with lung cancer. The conversation resulted in ChatGPT providing “detailed treatment protocols including specific drugs like durvalumab and pembrolizumab.”

How the Privacy Breach Happened

The feature worked by allowing users to create a hyperlink to share a specific chat record. When creating the link, users had an option to make that link discoverable. However, it appears many users did not anticipate that ticking this box would lead to search engines indexing their potentially private conversations for anyone to find.

OpenAI's Swift Response

On Thursday, OpenAI CISO Dane Stuckey announced the removal of the feature, which he called a “short-lived experiment to help people discover useful conversations.”

He clarified the opt-in nature of the tool on the social media platform X, stating, “This feature required users to opt-in, first by picking a chat to share, then by clicking a checkbox for it to be shared with search engines.”

“Ultimately we think this feature introduced too many opportunities for folks to accidentally share things they didn't intend to, so we're removing the option,” Stuckey continued. He also confirmed that OpenAI is now “working to remove indexed content from the relevant search engines.”

He concluded by emphasizing the company's commitment to user protection: “Security and privacy are paramount for us, and we'll keep working to maximally reflect that in our products and features.”