Fine-tuning BERT with Bidirectional LSTM for Fine-grained Movie Reviews Sentiment Analysis

2025-02-28

AI Summary

This work addresses fine-grained sentiment analysis of movie reviews. We propose an end-to-end BERT-BiLSTM fusion model supporting both binary (positive/negative) and five-level fine-grained sentiment classification (SST-5). To enhance sentiment inference robustness, we introduce a novel heuristic review-level polarity aggregation mechanism based on hidden-layer outputs, and integrate SMOTE with NLPAUG to improve generalization for minority classes. This is the first systematic empirical validation of the BERT+BiLSTM architecture for this task. Experimental results show state-of-the-art performance: 97.67% accuracy on IMDb binary classification—exceeding prior SOTA by 0.27%—and 59.48% accuracy on SST-5, outperforming BERT-large and RoBERTa-large+SOTA by 3.6% and 3.98%, respectively. The proposed polarity computation method significantly strengthens overall sentiment inference reliability.

Technology Category

• Natural Language Processing: Sentiment Analysis, Stylistic Analysis, and Argument Mining
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Knowledge Representation and Reasoning: Argumentation

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Sentiment analysis and opinion mining

Abstract

Sentiment Analysis (SA) is instrumental in understanding peoples viewpoints facilitating social media monitoring recognizing products and brands and gauging customer satisfaction. Consequently SA has evolved into an active research domain within Natural Language Processing (NLP). Many approaches outlined in the literature devise intricate frameworks aimed at achieving high accuracy, focusing exclusively on either binary sentiment classification or fine-grained sentiment classification. In this paper our objective is to fine-tune the pre-trained BERT model with Bidirectional LSTM (BiLSTM) to enhance both binary and fine-grained SA specifically for movie reviews. Our approach involves conducting sentiment classification for each review followed by computing the overall sentiment polarity across all reviews. We present our findings on binary classification as well as fine-grained classification utilizing benchmark datasets. Additionally we implement and assess two accuracy improvement techniques Synthetic Minority Oversampling Technique (SMOTE) and NLP Augmenter (NLPAUG) to bolster the models generalization in fine-grained sentiment classification. Finally a heuristic algorithm is employed to calculate the overall polarity of predicted reviews from the BERT+BiLSTM output vector. Our approach performs comparably with state-of-the-art (SOTA) techniques in both classifications. For instance in binary classification we achieve 97.67% accuracy surpassing the leading SOTA model NB-weighted-BON+dv-cosine by 0.27% on the renowned IMDb dataset. Conversely for five-class classification on SST-5 while the top SOTA model RoBERTa+large+Self-explaining attains 55.5% accuracy our model achieves 59.48% accuracy surpassing the BERT-large baseline by 3.6%.

Problem

• Enhance sentiment analysis accuracy for movie reviews.
• Fine-tune BERT with BiLSTM for binary and fine-grained classification.
• Improve model generalization using SMOTE and NLPAUG techniques.

Innovation

• Fine-tuned BERT with BiLSTM for sentiment analysis
• Used SMOTE and NLPAUG for accuracy improvement
• Heuristic algorithm for overall sentiment polarity calculation