Advanced Machine Learning Techniques for Social Support Detection on Social Media

2025-01-06

AI Summary

This study addresses the fine-grained identification and classification of social support behaviors in social media, aiming to accurately determine (i) the presence of supportive content, (ii) its target (individual vs. group), and (iii) its specific support type (e.g., LGBTQ+, gender, religious). We propose the first multi-task classification framework integrating zero-shot large language model (LLM) inference (GPT-3/4/4-o) with a K-means clustering–driven data balancing strategy. Compared to a traditional psycholinguistic + TF-IDF baseline, our approach achieves 0.4% and 0.7% improvements in macro-F1 for target identification (Task II) and support-type classification (Task III), respectively; Transformer-based models significantly outperform classical machine learning methods. The core contribution lies in synergistically incorporating zero-shot generalization capability and unsupervised data balancing into hierarchical social support classification—effectively mitigating challenges of annotation scarcity and severe class imbalance.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Natural Language Processing: Text Classification & Sentiment Analysis
• Application Domains: Humanities & Computational Social Science

Application Category

• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

The widespread use of social media highlights the need to understand its impact, particularly the role of online social support. This study uses a dataset focused on online social support, which includes binary and multiclass classifications of social support content on social media. The classification of social support is divided into three tasks. The first task focuses on distinguishing between supportive and non-supportive. The second task aims to identify whether the support is directed toward an individual or a group. The third task categorizes the specific type of social support, grouping it into categories such as Nation, LGBTQ, Black people, Women, Religion, and Other (if it does not fit into the previously mentioned categories). To address data imbalances in these tasks, we employed K-means clustering for balancing the dataset and compared the results with the original unbalanced data. Using advanced machine learning techniques, including transformers and zero-shot learning approaches with GPT3, GPT4, and GPT4-o, we predict social support levels in various contexts. The effectiveness of the dataset is evaluated using baseline models across different learning approaches, with transformer-based methods demonstrating superior performance. Additionally, we achieved a 0.4% increase in the macro F1 score for the second task and a 0.7% increase for the third task, compared to previous work utilizing traditional machine learning with psycholinguistic and unigram-based TF-IDF values.

Problem

• Social Media
• Support Behavior Classification
• Diverse Populations

Innovation

• Machine Learning
• K-means Clustering
• Transformers