From TF-IDF to Transformers: A Comparative and Ensemble Approach to Sentiment Classification

📅 2026-05-21
📈 Citations: 0
✨ Influential: 0
📄 PDF
🤖 AI Summary
This study addresses the challenges of contextual understanding and implicit sentiment recognition in movie review sentiment classification by systematically evaluating a range of approaches, from traditional statistical models—such as TF-IDF combined with Naive Bayes, Logistic Regression, SVM, and LightGBM—to modern deep learning architectures, including LSTM, DistilBERT, and RoBERTa. Building upon this comparative analysis, the authors propose a soft voting–based ensemble model to leverage the complementary strengths of individual classifiers. Experimental results on the IMDb dataset demonstrate that the RoBERTa single model achieves an accuracy of 93.02%, while the ensemble model further enhances overall performance, yielding superior results in terms of both accuracy and F1 score. These findings confirm the effectiveness and robustness of ensemble strategies for sentiment analysis tasks.
📝 Abstract
Sentiment analysis, also referred to as opinion mining, primarily tries to extract opinion from any text-based data. In the context of movie reviews and critics, sentimental analysis can be a helpful tool to predict whether a movie review is generally positive or negative. It can be difficult for the ML models to understand the context or metaphysical sentiment accurately, as ML models rely largely on statistical word representations. The objective of this paper is to examine and categorise movie reviews into positive and negative sentiments. Diverse machine learning models are considered in doing so, and Natural Language Processing (NLP) methodologies are employed for data preprocessing and model assessment. The IMDb dataset is used. Specifically, Naive Bayes, Logistic Regression, Support Vector Machines (SVM), LightGBM, LSTM, and transformer-based models such as RoBERTa and DistilBERT were evaluated. After a lot of testing with accuracy, precision, recall, F1-score, and ROC-AUC, RoBERTa performed better than all the other models, with an accuracy of 93.02%. A soft voting ensemble that combined all the models also improved classification performance, showing that model ensembling works well for sentiment analysis.
Problem

Research questions and friction points this paper is trying to address.

sentiment analysis
movie reviews
opinion mining
sentiment classification
natural language processing
Innovation

Methods, ideas, or system contributions that make the work stand out.

Sentiment Analysis
Transformer Models
Model Ensembling
RoBERTa
IMDb Dataset
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
No related jobs found.
D
Dip Biswas Shanto
School of Computer Engineering, KIIT Deemed to be University, Bhubaneswar, India
M
Mitali Yadav
School of Computer Engineering, KIIT Deemed to be University, Bhubaneswar, India
P
Prajwal Panth
School of Computer Engineering, KIIT Deemed to be University, Bhubaneswar, India
S
Suresh Chandra Satapathy
School of Computer Engineering, KIIT Deemed to be University, Bhubaneswar, India