Beyond the Human Signature: Transformer-Based Detection of AI-Generated Phishing Emails and Social Engineering Attacks

Abstract

The emergence of large language models capable of producing fluent, grammatically flawless prose has quietly dismantled one of the oldest heuristics in email security: the assumption that clumsy language betrays a phishing attempt. This paper synthesizes recent scholarship on transformer-based detection of phishing emails and social engineering messages, with particular attention to the harder problem of identifying content generated or substantially rewritten by large language models. Adopting a theoretical and integrative research design rather than a new empirical experiment, the study draws on peer-reviewed and reputable preprint literature published mainly between 2022 and 2026 to trace how encoder-based transformers such as BERT and RoBERTa, distilled variants such as DistilBERT, and emerging large language model detectors have been applied to this problem, and to identify where their performance claims break down under adversarial or out-of-distribution conditions. The synthesis finds that transformer architectures deliver consistently high accuracy on established benchmark corpora, frequently exceeding ninety-eight percent, yet this performance depends heavily on corpus composition and degrades measurably when models encounter phishing campaigns authored by generative systems rather than by the human attackers represented in legacy datasets. The paper develops a conceptual comparison of model families, spanning encoder-only transformers, distilled architectures, hybrid neural designs, large language model detectors, and stylometric or perplexity-based approaches, mapping the strengths and failure modes associated with each. The discussion situates these findings within a broader argument that detection can no longer rest on linguistic anomaly alone and must instead incorporate behavioral, contextual, and provenance signals alongside textual analysis. The paper closes with recommendations for adversarially robust benchmarking and human-in-the-loop deployment.