How Hackers Use Agentic AI to Advance Social Engineering

Every day, cybercriminals send 3.4 billion phishing emails, designed to mimic trusted sources. (Source) That’s over a trillion phishing emails per year, fueling fraud, breaches, and financial losses.

Now, AI-powered phishing is making these attacks faster, more scalable, and nearly undetectable. Hackers are leveraging agentic AI—a form of AI that autonomously gathers intelligence, crafts hyper-personalized lures, and executes large-scale attacks with minimal effort.

This blog explores how AI is reshaping phishing, why traditional defenses are failing, and what organizations must do to stay ahead of AI-driven threats.

AI vs. Human Phishing: A Game-Changing Reality

A recent study by researchers from Harvard Kennedy School and Avant Research Group tested how effective AI-generated phishing emails are compared to those written by humans. (Source) They sent four types of emails to 101 real participants and measured the click rates:

• Basic scam emails (used as a control group): Only 12% of people clicked on these generic, poorly written emails.
• Phishing emails written by human experts: 54% of people clicked, showing that well-crafted scams are much more convincing.
• Phishing emails written entirely by AI: 54% of people clicked, proving that AI can already match human experts in deception.
• AI-generated phishing emails with minor human edits: 56% of people clicked, slightly outperforming both AI-only and human-written attempts.

Why does this matter?

These results are shocking because they show that AI can now create phishing emails just as convincingly as skilled cybercriminals—and sometimes even better. This means hackers no longer need to be experts at writing scams. Instead, they can use AI to generate unlimited, high-quality phishing emails in seconds, making attacks easier, faster, and more widespread.

So, why is AI-powered phishing so effective? Let’s break it down.

How AI is Revolutionizing Social Engineering

AI is transforming phishing from a manual process into a fully automated, intelligent attack system. Unlike traditional scams, AI-powered phishing continuously learns, adapts, and personalizes attacks at an unprecedented scale. By leveraging automation and advanced data analysis, AI can craft hyper-realistic phishing attempts that are nearly impossible to detect.

1. AI Learns and Self-Improves Faster Than Humans

Traditional hackers refine their tactics through trial and error, but AI can optimize phishing emails instantly by analyzing real-time data. It examines:

• Click rates and response patterns to craft more convincing messages.
• Psychological triggers and linguistic nuances to manipulate targets more effectively.

Since AI learns at machine speed, phishing techniques that fail today can be perfected tomorrow—making attacks constantly evolve and improve.

2. AI Doesn’t Just Write Emails—It Conducts Automated Target Research

AI doesn't just generate emails—it gathers intelligence on targets before launching attacks. Through Open-Source Intelligence (OSINT), AI can:

• Scan social media, company websites, and public records to profile individuals.
• Identify job roles, communication styles, and interests to personalize attacks.
• Mimic familiar language and behaviors, making phishing emails almost indistinguishable from legitimate communication.

With this data-driven targeting, AI phishing is no longer a guessing game—it’s precision-engineered deception.

3. AI Removes Traditional Barriers to Phishing

Phishing once required technical skill, research, and effort—AI has removed these obstacles, making attacks effortless and highly effective.

• AI generates tailored phishing emails in seconds, adapting to a target’s job role, industry, and online activity.
• It scans social media and company directories to insert real details, like colleagues’ names or recent events, increasing credibility.
• It writes fluently in any language, avoiding grammar mistakes that typically expose scams.
• It runs continuously, refining tactics based on response rates, something human attackers can’t do at scale.

AI has transformed phishing into an automated, precision-targeted cyber threat—faster, smarter, and harder to detect.

4. AI Bypasses Security Guardrails

AI models are designed with built-in safety measures to prevent abuse, but cybercriminals have already found ways to bypass them:

• Rewording prompts to trick AI into generating phishing content.
• Using agent-based AI, which autonomously collects target data and launches phishing attacks.
• Altering phishing emails in real time to evade detection by spam filters and security systems.

With these methods, hackers stay ahead of AI safety measures, making AI-driven phishing one of the biggest cybersecurity threats today.

The Economics of AI-Powered Phishing

AI has made phishing cheaper, faster, and more profitable, giving cybercriminals a major advantage.

• AI-driven phishing is up to 50x more profitable than traditional methods by increasing success rates while cutting costs.
• It eliminates the need for technical skills, allowing even inexperienced attackers to launch advanced phishing campaigns.
• Automation enables mass attacks with minimal effort, making phishing more widespread and harder to stop.

As AI continues to lower the barriers to cybercrime, we can expect a surge in phishing attacks across all industries.

AI Enables Multi-Channel Phishing Attacks

Phishing is no longer just about deceptive emails—AI now powers attacks across multiple channels, making scams harder to detect. Cybercriminals use AI to:

• Create fake online personas to manipulate employees, executives, or IT staff.
• Generate deepfake voice and video impersonations to bypass identity verification.
• Develop phishing websites that perfectly mimic real login portals, even bypassing Multi-Factor Authentication (MFA).

This new wave of AI-driven deception is blurring the line between real and fake, making it increasingly difficult for individuals to recognize threats.

The Future of Phishing: What Comes Next?

AI-powered phishing is rapidly advancing, with attacks becoming more personalized, automated, and nearly impossible to distinguish from real communications. Traditional security measures struggle to keep up with the constant adaptation of AI and the generation of new attack strategies. To stay ahead, organizations must embrace AI-driven defenses and rethink how they approach cybersecurity.

1. AI-Powered Attacks Will Become Hyper-Personalized

AI-driven phishing is moving beyond generic scams, evolving into highly targeted attacks that exploit real-time data. Future threats will:

• Analyze live social media activity and business emails to craft ultra-personalized lures.
• Use deepfake technology to impersonate colleagues, executives, or clients in emails and video calls.
• Apply behavioral psychology and AI-generated emotional triggers to manipulate targets more effectively.

2. Traditional Security Measures Will Struggle to Keep Up

Phishing detection tools rely on pattern recognition, but AI constantly generates new attack variations that evade security filters. Organizations will face:

• AI-crafted phishing messages that never repeat, making them harder to blacklist.
• Sophisticated bypass techniques, allowing AI-driven phishing sites to evade web security checks.
• Automated, multi-channel attacks, combining emails, SMS, and voice phishing (vishing) for maximum deception.

3. Organizations Must Use AI to Fight AI

Since AI is driving a new era of cyber threats, security teams must leverage AI-driven defense mechanisms, such as:

• Adaptive phishing detection systems that analyze behavioral patterns instead of fixed rules.
• AI-powered security awareness training, simulating real-world attack scenarios to build employee resilience.
• Beyond-password authentication methods, like biometric verification and continuous identity monitoring, to prevent credential-based attacks.

AI is rewriting the rules of phishing, and organizations must act now to stay ahead of this rapidly evolving threat.

How Keepnet Can Protect Against AI-Powered Phishing

As AI-driven phishing evolves, organizations need advanced simulations, adaptive training, and rapid response solutions. Keepnet’s AI-powered tools provide a strong defense against these threats.

• AI-Driven Phishing Simulations – Keepnet’s Phishing Simulator replicates AI-generated attacks, helping employees detect and avoid sophisticated scams.
• Adaptive Security Awareness Training – Offers 2,100+ training materials from 15+ providers in 36+ languages, using AI-driven phishing simulations and a behavior change model to deliver personalized, multilingual training.
• Automated Threat Response with Incident Responder – Detects, analyzes, and mitigates AI-powered phishing threats in real time, responding 48.6x faster than manual processes.

With AI-driven simulations, tailored training, and automated response, Keepnet helps organizations stay ahead of emerging phishing threats.

Why AI-Powered Phishing Requires a New Defense Strategy

AI-driven phishing is evolving beyond traditional scams, using automation, deepfakes, and real-time data to deceive even the most cautious employees. Standard security measures can’t keep up, making AI-powered defenses essential.

Check out the Keepnet Extended Human Risk Management Platform to simulate real AI-driven phishing attacks, train employees with adaptive learning, and automate phishing response—stopping threats before they cause damage.