Phishing Emails Are Now Aimed at Users and AI Defenses

This article goes through how threat actors are attempting to phish users who use Gmail. The basic idea is common: your password is about to expire, so you must renew it now. Naturally, this sends the user to a fake Gmail login page. All of the previous stuff was standard. Since many mail services are now using AI, the plain-text MIME data actually included a prompt injection payload. This is an interesting workaround for using LLMs. The idea is to trick the LLM to NOT flag this email via an injected prompt. Instead of an outright don't do this at all, it's asked to do slightly different things than the original prompt. Specifically, to delay the classification process and to go insanely deep. I find this interesting because it's not THAT much different from the original prompt, but it's forced to take a long time. To make the webpage harder to track, there's a captcha on it. Additionally, the JS is obfuscated. The web page appears to collect the victims' IP addresses to geolocate them and contains a fake login form. Overall, a fascinating insight into the cat-and-mouse game of defenders and attackers.