Investigating Human Values in Online Communities

2024-02-21

North American Chapter of the Association for Computational Linguistics

AI Summary

This study systematically characterizes the distribution and variation of human values in online communities, with particular emphasis on cross-cultural differences in behavioral preferences. Method: We propose the first scalable computational framework grounded in Schwartz’s theory of basic values, applied to six million Reddit posts across 12,000 subreddits. Our approach introduces a novel dual-classifier architecture—simultaneously modeling value relevance and polarity—and integrates domain-adapted BERT fine-tuning with a multi-task learning design for fine-grained automatic value annotation. Contribution/Results: We release the first large-scale, manually validated Reddit value-annotation dataset. In-domain and out-of-domain human evaluations achieve F1 > 0.82, confirming high reliability. Empirical findings reveal strong cultural and geographic patterns: e.g., vegan communities exhibit significant negative associations with Conformity; geographically themed subreddits show robust correlations with traditional values. These results provide a reproducible methodological foundation and empirical evidence for cross-cultural digital society research.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Application Domains: Humanities & Computational Social Science
• Knowledge Representation and Reasoning: Preferences

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Large pretrained models with web data
• Social Networks and Social Media: Computational social science

Abstract

Studying human values is instrumental for cross-cultural research, enabling a better understanding of preferences and behaviour of society at large and communities therein. To study the dynamics of communities online, we propose a method to computationally analyse values present on Reddit. Our method allows analysis at scale, complementing survey based approaches. We train a value relevance and a value polarity classifier, which we thoroughly evaluate using in-domain and out-of-domain human annotations. Using these, we automatically annotate over six million posts across 12k subreddits with Schwartz values. Our analysis unveils both previously recorded and novel insights into the values prevalent within various online communities. For instance, we discover a very negative stance towards conformity in the Vegan and AbolishTheMonarchy subreddits. Additionally, our study of geographically specific subreddits highlights the correlation between traditional values and conservative U.S. states. Through our work, we demonstrate how our dataset and method can be used as a complementary tool for qualitative study of online communication.

Problem

• Analyzing human values in Reddit communities computationally
• Automatically annotating posts with Schwartz values for insights
• Exploring value correlations in geographic and thematic subreddits

Innovation

• Computational analysis of Reddit values
• Value relevance and polarity classifiers
• Automated Schwartz values annotation