
Does anyone remember the first time they learned about phishing attacks and how they work? I remember learning about these attack tactics early in my career as a Security Engineer from my peers and various blogs that detailed the attack path. We traded stories, successes, and failures and built up a store of knowledge that we rely on to this day. In this blog, I will address how phishing has evolved into AI phishing and what security professionals are doing about it.
What is phishing?
Phishing refers to fake e-mails that hackers send to try to trick recipients into either clicking on a dangerous link, downloading a malicious file, or replying with information a hacker can use to gain access to the recipient’s accounts. Hackers often phish businesses with these e-mails because they can reliably count on at least one person in a company of hundreds or thousands to unthinkingly fall for the bait.
In the early days, the basic advice we gave to avoid being phished was along the lines of:
- Check the sender’s address: if it is made up of random characters, it’s probably fake.
- Hover over the link to make sure it goes to where it says.
- If it’s an offer that seems too good to be true, it probably is!
And the big one we always shared with our teams:
- Check for grammar or blatant spelling mistakes.
Of course, attackers always figure out how to go around the advice we’re sending out. This game of “cat and mouse” continues as always: defenders come up with a way to stop attacks, and the attackers figure out a way around it. Rinse and repeat.
How has the security industry combatted phishing in the past?
The security industry developed tooling that looks for the indicators mentioned above. It started weeding out bad sending domains, blocking attachments of certain types, and “exploding” links in virtual machines before final mail delivery to help protect the recipient from malicious intent and means.
The industry also got better at security and awareness training with fantastic content created to help everyone spot a phishing attack. In some cases, organizations require everyone to complete annual training (or more frequently!) on this very subject.
Then attackers got better at phishing again, of course, with new tactics and techniques like registering look-alike domains, using expired domains instead of new ones, and using more imaginative evasion techniques to allow more of their links and attachments to get through. The requisite controls and training resources responded in kind, and the cycle continued.
Then, Generative AI (GenAI) solutions started to appear on the scene. Attackers have leveraged these technologies to generate phishing content more effectively and rapidly.
Read more: Strengthening security in an AI-driven world
How effective is AI phishing?
Recent research performed by Bruce Schneier et al. shows that using LLMs or GenAI solutions in phishing campaigns leads to a 60% success rate and an upwards of 95% cost reduction in the overall attack chain related to phishing.
In short, it is not only getting easier to create effective AI phishing campaigns, but also getting WAY cheaper! I recently received a spam message that I thought was likely generated using an LLM based on how the content was written and presented. Here’s a screenshot of the message:

There are a few hallmarks of a phishing e-mail—which sent it to my spam folder—but overall, the content is pretty good. It’s topical, relevant, and honestly, mean enough to create the type of psychological response that an attacker wants to exploit: “Oh no, I’m being laid off. What’s this receipt?!?”
I thought, “This really does look like something that ChatGPT or something would have created,” so I decided to see if I could generate a similar message using another LLM, Microsoft Copilot.
With just five prompts to clarify some of the points, here is what I was able to create:

Here are the specific prompts I used in order:
- Can you help me draft an e-mail letting a person with our company know that their position has been eliminated due to increased tax tariffs?
- How should I address the employee’s concerns?
- Can you provide examples of empathetic responses?
- Can you help me draft a specific message using these examples?
- Can you sign it with a more appropriate message from the HR department?
Copilot just suggested prompts 2-4, so all I had to write were two of the five prompts, and I created a pretty convincing AI phishing e-mail that is very similar to my previous example in minutes.
From here, I would have to register a domain, generate a payload, and e-mail whomever I wanted. I wouldn’t necessarily be able to automate all of that with ChatGPT, Claude, or Copilot due to specific programmatic guardrails each solution has in place, but there are models out there that do not have the same controls. These models can be convinced to generate malicious payloads, landing pages, and delivery mechanisms and even send e-mails for an attacker (I won’t link all that here for safety reasons, but they are not that hard to find).
Watch: AI-powered secrets to outsmart cyber criminals
There is good news in the sea of AI phishing
While AI phishing attempts do not initially seem easily defendable, there are ways to make sure your organization is prepared to combat this new cost-effective attack route in addition to the “traditional” methods of multi-factor authentication, anti-phishing protection technologies, and effective patch management:
- Continued security and awareness training. Play out scenarios using LLM-generated content in a safe manner that will benefit your organization before an attacker has a chance to test you!
- AI-driven security tools. I know everyone says they have AI-driven security tools, but using the same tooling as the malicious actors is one of the best ways to combat their tactics. Find a reputable vendor and leverage their tooling and expertise!
- Regular security assessments. Conducting regular security assessments can help identify vulnerabilities and ensure that your defenses are up to date. Let our assessments team, with the aid of AI-enhanced offensive tools, perform a custom assessment of your organization and then help provide recommendations to remediate any findings!
Phishing will persist as long as humans need computers—that is to say, forever—which means that security professionals must be just as persistent combatting it even as the attackers put more sophisticated AI phishing tactics to work for them.
Read more: New frontiers in security: the rise of Zero Trust and AI endpoint tools