Automated Research Article Classification and Recommendation Using NLP and ML

2025-10-06

AI Summary

To address low academic discovery efficiency caused by information overload in digital libraries, this paper proposes an end-to-end intelligent classification and recommendation framework for scholarly papers. Methodologically, it systematically compares text representation techniques—including TF-IDF, Sentence-BERT, and Universal Sentence Encoder (USE)—combined with classical classifiers (logistic regression, SVM, and Naïve Bayes), and integrates a cosine-similarity-driven collaborative recommendation module. Experiments on the large-scale arXiv dataset show that the TF-IDF + logistic regression configuration achieves 69% classification accuracy, while the recommendation module significantly improves retrieval precision and coverage of relevant literature. The primary contribution is a reproducible, scalable, lightweight hybrid NLP–ML framework that balances high classification performance with practical recommendation utility, offering an effective technical pathway for academic search engines and personalized knowledge services.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Representation Learning
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text

Abstract

In the digital era, the exponential growth of scientific publications has made it increasingly difficult for researchers to efficiently identify and access relevant work. This paper presents an automated framework for research article classification and recommendation that leverages Natural Language Processing (NLP) techniques and machine learning. Using a large-scale arXiv.org dataset spanning more than three decades, we evaluate multiple feature extraction approaches (TF--IDF, Count Vectorizer, Sentence-BERT, USE, Mirror-BERT) in combination with diverse machine learning classifiers (Logistic Regression, SVM, Naïve Bayes, Random Forest, Gradient Boosted Trees, and k-Nearest Neighbour). Our experiments show that Logistic Regression with TF--IDF consistently yields the best classification performance, achieving an accuracy of 69%. To complement classification, we incorporate a recommendation module based on the cosine similarity of vectorized articles, enabling efficient retrieval of related research papers. The proposed system directly addresses the challenge of information overload in digital libraries and demonstrates a scalable, data-driven solution to support literature discovery.

Problem

• Automating research article classification using NLP and ML
• Recommending relevant papers through cosine similarity matching
• Addressing information overload in scientific literature discovery

Innovation

• Uses NLP and ML for article classification
• Evaluates multiple feature extraction methods
• Recommends articles via cosine similarity vectors