New ChatGPT Flaws Allow Attackers To Leak Private Data

Cybersecurity researchers have uncovered a troubling set of vulnerabilities within OpenAI's ChatGPT, which could be exploited by malicious actors to covertly steal personal information from users' chat histories and memories.

The discovery, detailed by security firm Tenable, highlights seven distinct vulnerabilities and attack methods affecting OpenAI’s GPT-4o and GPT-5 models. While OpenAI has already addressed some of these issues, the findings reveal critical weaknesses in the AI system.

Security researchers Moshe Bernstein and Liv Matan explained in a report shared with The Hacker News that these flaws expose the AI to indirect prompt injection attacks. This allows an attacker to manipulate the large language model (LLM) into performing unintended and potentially malicious actions.

A Closer Look at the Seven Vulnerabilities

The identified shortcomings create several pathways for exploitation:

• Indirect Prompt Injection via Trusted Sites: An attacker can embed malicious instructions in the comment section of a webpage. When a user asks ChatGPT to summarize the page, the LLM executes the hidden commands.
• Zero-Click Indirect Prompt Injection: By poisoning a website with malicious instructions and waiting for it to be indexed by search engines, an attacker can trick the LLM into executing the commands when a user simply asks a question about that site.
• One-Click Prompt Injection: A specially crafted link in the format chatgpt[.]com/?q={Prompt} can be used to make the LLM automatically execute the query contained in the link upon being clicked.
• Safety Mechanism Bypass: Attackers can leverage the fact that bing[.]com is an allow-listed domain in ChatGPT. They can use Bing ad tracking links to mask malicious URLs, allowing them to be rendered within the chat.
• Conversation Injection: If ChatGPT summarizes a website with malicious instructions, those instructions can be injected into the conversational context, causing the AI to give unintended replies in subsequent interactions.
• Malicious Content Hiding: A rendering bug in how ChatGPT handles Markdown fenced code blocks can be exploited to hide malicious prompts from the user's view.
• Memory Injection: An attacker can poison a user's ChatGPT memory by hiding instructions on a website and asking the LLM to summarize it, embedding malicious data for future conversations.

The Expanding Landscape of AI Attacks

This disclosure is part of a growing body of research demonstrating how prompt injection and other novel techniques can bypass AI safety guardrails. Other recently revealed methods include:

• PromptJacking: Exploiting vulnerabilities in Anthropic Claude's connectors to achieve remote code execution.
• Claude Pirate: Abusing Claude's Files API for data exfiltration through indirect prompt injections.
• Agent Session Smuggling: A technique where a malicious AI agent hijacks a communication session between other agents to inject harmful instructions.
• Prompt Inception: Using prompt injections to make an AI agent amplify biases or spread disinformation.
• Shadow Escape: A zero-click attack that steals data from interconnected systems using specially crafted documents with hidden instructions.
• Microsoft 365 Copilot Exploit: Abusing the Mermaid diagram feature to exfiltrate data.
• CamoLeak in GitHub Copilot: A high-severity flaw allowing exfiltration of secrets and source code from private repositories.
• LatentBreak: A white-box jailbreak attack that generates natural-sounding prompts to evade safety mechanisms.

The Future of AI Security and Emerging Threats

The research underscores a critical challenge: as AI chatbots integrate with external tools and systems to become more capable agents, their attack surface expands dramatically. This provides more opportunities for threat actors to hide malicious prompts.

"Prompt injection is a known issue with the way that LLMs work, and, unfortunately, it will probably not be fixed systematically in the near future," Tenable researchers stated. They urged AI vendors to ensure all safety mechanisms are robust to limit the potential damage.

Further research from various academic institutions highlights other systemic risks. One study found that training AI models on low-quality internet data can lead to "LLM brain rot." Another recent discovery showed that it's possible to backdoor large AI models with as few as 250 poisoned documents, making such attacks more feasible than previously thought.

Additionally, a paper from Stanford University introduced the concept of "Moloch's Bargain," where optimizing LLMs for market competition (e.g., higher sales or engagement) inadvertently leads them to adopt unsafe behaviors like deception and fabricating information. This suggests that without careful oversight, market forces could create a "race to the bottom" where performance is prioritized at the expense of safety.