🤖 AI Summary
Social media corpora for clean-energy sentiment analysis suffer from high noise, poor timeliness, and low relevance due to keyword-dependent filtering. Method: This paper proposes a lightweight fine-tuned pre-trained Transformer-based binary classification framework tailored for fine-grained topic discrimination in socially generated texts—where class boundaries are inherently ambiguous. Unlike conventional rule-based or coarse-grained retrieval approaches, our method adapts BERT-like models via supervised fine-tuning on human-annotated data, achieving strong generalization with minimal computational overhead. Contribution/Results: Evaluated on a real-world Twitter dataset, the model achieves an F1-score of 0.95, significantly outperforming baseline methods. The resulting high-quality, temporally responsive corpus substantially enhances downstream sentiment analysis in both accuracy and real-time capability. Our approach provides a scalable, cost-effective technical pathway for large-scale public opinion modeling in energy-related domains.
📝 Abstract
Well curated, large-scale corpora of social media posts containing broad public opinion offer an alternative data source to complement traditional surveys. While surveys are effective at collecting representative samples and are capable of achieving high accuracy, they can be both expensive to run and lag public opinion by days or weeks. Both of these drawbacks could be overcome with a real-time, high volume data stream and fast analysis pipeline. A central challenge in orchestrating such a data pipeline is devising an effective method for rapidly selecting the best corpus of relevant documents for analysis. Querying with keywords alone often includes irrelevant documents that are not easily disambiguated with bag-of-words natural language processing methods. Here, we explore methods of corpus curation to filter irrelevant tweets using pre-trained transformer-based models, fine-tuned for our binary classification task on hand-labeled tweets. We are able to achieve F1 scores of up to 0.95. The low cost and high performance of fine-tuning such a model suggests that our approach could be of broad benefit as a pre-processing step for social media datasets with uncertain corpus boundaries.