AI Outperforms Human Red Teams in Phishing Assessments

Artificial intelligence has advanced beyond its initial capabilities in games like chess and Go and is now effectively outperforming human red teams in phishing simulations on a large scale, according to findings from cybersecurity training provider Hoxhunt.

See Also: Live Webinar | AI-Powered Defense Against AI-Driven Threats

The analysis reveals that Hoxhunt’s proprietary AI spear phishing agent, code-named JKR (for Joker), has achieved a 24% higher success rate in crafting phishing attacks than human counterparts—an impressive recovery from a 31% deficit in a similar test conducted in 2023.

Hoxhunt characterized this development as a “Skynet moment for social engineering,” referring to the AI antagonist from the Terminator series. The firm stated that this performance demonstrates that AI agents can efficiently generate superior spear phishing attacks at scale.

JKR’s effectiveness stems from its real-time optimization of prompts and output, allowing the AI to personalize messages based on user-specific variables such as role and location, thereby creating highly tailored emails for numerous enterprise users.

The organization anticipates that the phishing-as-a-service sector will witness a significant transition towards widespread adoption of AI-driven spear phishing agents. Consequently, the overall quality of mass phishing campaigns is expected to reach levels currently associated with targeted spear phishing attacks.

In March, the Anti-Phishing Working Group (APWG) reported a significant uptick in global phishing emails following a period of decline the previous year. The APWG noted that nearly one million phishing websites were identified within the last quarter of 2024 alone. Furthermore, U.S. officials have raised alarms about a surge in smishing attempts linked to deceptive toll collection messages, revealing that the campaigns often lack precise targeting.

“Phishing messages are typically sent to random phone numbers, sometimes reaching individuals who do not utilize toll roads or are located in incorrect states,” the APWG stated.

Experts in the field, like Casey Ellis, founder of Bugcrowd, emphasize that while humans possess unique creativity and adaptability, AI can execute tactics without rest, often outperforming human efforts when scope is prioritized. However, Amit Zimerman, co-founder and CPO at Oasis Security, warns against an overreliance on AI to combat AI-driven phishing, citing potential risks such as false positives and contextual misjudgments. The importance of human oversight remains paramount in understanding AI-generated outcomes and making informed decisions.