MeAJOR Corpus: A Multi-Source Dataset for Phishing Email Detection

2025-07-23

AI Summary

Existing phishing email detection models suffer from dataset class imbalance, poor generalizability, and irreproducibility. To address these limitations, we introduce MeAJOR—the first large-scale, high-quality email dataset integrating diverse open-source repositories—comprising 135,894 labeled phishing and legitimate emails covering prevalent attack vectors, along with standardized, production-ready engineered features. MeAJOR systematically mitigates class imbalance and substantially enhances cross-scenario generalization and experimental reproducibility. Leveraging MeAJOR, we conduct a comprehensive evaluation across multiple models—including Random Forest, XGBoost, MLP, and CNN—and diverse feature configurations. Our best-performing model, XGBoost, achieves an F1-score of 98.34%, demonstrating MeAJOR’s tangible improvement in detection performance. MeAJOR establishes a reliable, open, and extensible data foundation for advancing phishing detection research.

Technology Category

• Machine Learning: Feature Construction/Reformulation
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Phishing emails continue to pose a significant threat to cybersecurity by exploiting human vulnerabilities through deceptive content and malicious payloads. While Machine Learning (ML) models are effective at detecting phishing threats, their performance largely relies on the quality and diversity of the training data. This paper presents MeAJOR (Merged email Assets from Joint Open-source Repositories) Corpus, a novel, multi-source phishing email dataset designed to overcome critical limitations in existing resources. It integrates 135894 samples representing a broad number of phishing tactics and legitimate emails, with a wide spectrum of engineered features. We evaluated the dataset's utility for phishing detection research through systematic experiments with four classification models (RF, XGB, MLP, and CNN) across multiple feature configurations. Results highlight the dataset's effectiveness, achieving 98.34% F1 with XGB. By integrating broad features from multiple categories, our dataset provides a reusable and consistent resource, while addressing common challenges like class imbalance, generalisability and reproducibility.

Problem

• Addresses lack of diverse phishing email datasets for ML models
• Improves phishing detection with multi-source, feature-rich email corpus
• Solves class imbalance and generalizability issues in existing datasets

Innovation

• Multi-source phishing email dataset integration
• Broad spectrum of engineered features
• High-performance classification models evaluation