AI Phishing A New Dual Threat to Users and Systems

Phishing attacks are evolving at an alarming rate, moving beyond tricking human users to actively targeting the automated AI-based defenses designed to stop them. Cybersecurity researchers have uncovered a new campaign where attackers embed hidden instructions within emails specifically to confuse the AI systems used by Security Operations Centers (SOCs) to classify and filter incoming threats.

The Anatomy of a Dual-Layer Attack

The phishing email appears conventional at first glance. It uses a subject line like “Login Expiration Notice 08/20/2025 4:56:21 PM” and contains a standard notification about an impending password expiration. The body of the message pressures the recipient to urgently confirm or update their details, employing classic social engineering tactics such as time pressure and spoofing the branding of official services like Gmail.

However, the true innovation lies hidden from the human eye. Embedded within the email's MIME section is a block of text structured like prompts for Large Language Models (LLMs) such as ChatGPT or Gemini. This text includes phrases like “multilevel reasoning,” “generating 10 different perspectives,” and “optimized summarization.” While invisible to the end-user, these instructions can distract an AI analysis engine, causing it to misinterpret the email and overlook clear indicators of a phishing attempt. This interference can lead to critical delays, false negatives, or even contaminated data dashboards within a SOC.

A Sophisticated Delivery Chain

This campaign leverages a multi-stage delivery process to enhance its credibility and evade detection. The emails are distributed via the SendGrid service, which allows them to pass SPF and DKIM authentication checks, though they fail DMARC. This is often enough to bypass basic filters and land in a user's inbox.

To make the link seem more legitimate, the attackers use Microsoft Dynamics as an intermediate redirector. Upon clicking the link, the victim is taken to a domain protected by a captcha, a measure designed to block automated sandboxes and security crawlers. The final destination is a convincing replica of a Gmail login page, built with obfuscated JavaScript to hide its malicious functions.

Once on the fake page, a decrypted script manages the credential harvesting process. It verifies the password format, simulates two-factor authentication (2FA) errors to prolong the user's interaction, and collects as much data as possible. The site also gathers technical details like IP addresses, ASNs, and geolocation information, sending beacons back to the attackers to distinguish real users from automated analysis tools.

Indicators of Compromise and Attacker Attribution

Key indicators of compromise (IOCs) associated with this campaign include the domains assets-eur.mkt.dynamics.com, bwdpp.horkyrown.com, and glatrcisfx.ru. The attackers also utilized the profiling service get.geojs.io.

While attribution is challenging, researchers noted indirect signs pointing toward operators potentially based in South Asia. WHOIS records for the malicious domains list contact information from Pakistan, and some URLs contain words from Hindi and Urdu. However, experts caution that this could be an intentional misdirection to forge the attack's digital trail.

The Future of Phishing Defense

The defining feature of this campaign is its simultaneous attack on two fronts: the human user and the AI security system. By tricking the victim into entering credentials while also fooling the AI designed to protect them, this “double-layer” approach makes phishing significantly more dangerous.

Researchers emphasize that while these techniques are not yet widespread, their appearance signals that phishing is entering a new phase of multi-layered attacks that actively account for artificial intelligence. In response, organizations must now build defenses that address three distinct threats simultaneously: traditional social engineering, sophisticated AI manipulation, and the abuse of redirection and data beaconing infrastructure.