Red Hot Cyber

Cybersecurity is about sharing. Recognize the risk, combat it, share your experiences, and encourage others to do better than you.
Search

AI Phishing is here! ChatGPT, click here now to avoid losing access!

Redazione RHC : 25 August 2025 15:40

Phishing attacks are becoming increasingly sophisticated and are now targeting not only users, but also automated AI-based defenses. Researchers have discovered a campaign in which attackers embed hidden instructions in emails to confuse the AI systems used by SOCs to classify and filter threats.

The email itself had a traditional look: the subject line was “Login Expiration Notice 08/20/2025 4:56:21 PM”, and the body was a notification about an impending password expiration to an email address with a request to urgently confirm or update the data. This technique relies on familiar elements of social engineering: time pressure, imitating official messages, and spoofing Gmail’s branding.

But the email contained something much more interesting: a block of text in the MIME section, written in the style of LLM prompts like ChatGPT or Gemini. It included references to “multilevel reasoning,” “generating 10 different perspectives,” and “optimized summarization.” These references are hidden from users, but when analyzing an email, the AI could be distracted by these instructions and miss obvious signs of phishing.

If these algorithms are related to process automation (tagging, escalation, ticket opening), such interference can lead to delays, false negatives, or contaminated SOC dashboards.

The distribution chain itself is a copy of the previous campaign with minor modifications. The emails were sent via SendGrid, passing SPF/DKIM but not DMARC, allowing them to bypass filters and access inboxes. The attackers used Microsoft Dynamics as an intermediate redirect to make the message more credible. The victim was then greeted by a domain with a captcha that blocked sandboxes and crawlers, and the final page mimicked a Gmail login form with obfuscated JavaScript.

The first-stage loader contained an encrypted AES-CBC cipher; the key and IV (the first 16 bytes of the block) were hidden in Base64. Once decrypted, a script was executed that controlled the fake login process: password verification, simulating 2FA errors, and prolonging the interaction to extort data. Furthermore, the site collected IP addresses, ASNs, and geolocations, and sent beacons to distinguish real users and for automated analysis.

Indicators of compromise included the domains assets-eur.mkt.dynamics.com, bwdpp.horkyrown.com, and glatrcisfx.ru, as well as access to the get.geojs.io profiling service. Experts detected several indirect signs that the operators were potentially affiliated with South Asia. The WHOIS records of the attacking domains contain contact information from Pakistan, and the URLs contain words characteristic of Hindi and Urdu (“tamatar” (“tomato”), “chut” (an obscene word), which indicates the attack’s possible origin in South Asia, although researchers point to the possibility of trace forgery.

The main difference between this campaign and previous ones is the explicit attempt to attack two targets simultaneously: humans and artificial intelligence. The victim is tricked into entering credentials, and the AI system is fooled by embedded prompts. This “double layer” makes phishing much more dangerous: now not only users must protect themselves, but also the security tools themselves.

The researchers emphasize that These techniques are still rare, but their emergence demonstrates that phishing has entered the stage of“multi-layered attacks that take artificial intelligence into account.” Companies will now have to build defenses in three directions simultaneously: against social engineering, against AI manipulation, and against the abuse of redirection and beacon infrastructure.

Redazione
The editorial team of Red Hot Cyber consists of a group of individuals and anonymous sources who actively collaborate to provide early information and news on cybersecurity and computing in general.

Lista degli articoli