A Cross-Cultural Assessment of Human Ability to Detect LLM-Generated Fake News about South Africa

2025-11-21

AI Summary

This study investigates how cultural proximity influences the ability to detect AI-generated fake news, specifically examining performance differences between South African locals and non-locals in distinguishing authentic versus fabricated news about South Africa. Method: Using LLM-generated fake news samples, we conducted a controlled experiment combined with surveys, employing mixed-method analysis of behavioral responses and justification patterns from 89 participants. Contribution/Results: Cultural proximity exerts a dual effect: South African participants exhibited higher accuracy in identifying real news (40% vs. 52% deviation) but lower vigilance toward fake news (62% vs. 55% detection rate), yielding comparable overall discrimination bias (51% vs. 53%). Mechanistically, locals relied more on contextual knowledge for verification, whereas non-locals prioritized linguistic surface features. While cultural familiarity enhances true-information confirmation, it may induce trust bias that impairs fake-content detection. This work provides the first empirical evidence of culture’s “double-edged sword” effect in AI-generated misinformation detection, offering theoretical foundations and design implications for cross-cultural disinformation governance.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Application Domains: Misinformation & Fake News
• Humans and AI: Learning Human Values and Preferences

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

This study investigates how cultural proximity affects the ability to detect AI-generated fake news by comparing South African participants with those from other nationalities. As large language models increasingly enable the creation of sophisticated fake news, understanding human detection capabilities becomes crucial, particularly across different cultural contexts. We conducted a survey where 89 participants (56 South Africans, 33 from other nationalities) evaluated 10 true South African news articles and 10 AI-generated fake versions. Results reveal an asymmetric pattern: South Africans demonstrated superior performance in detecting true news about their country (40% deviation from ideal rating) compared to other participants (52%), but performed worse at identifying fake news (62% vs. 55%). This difference may reflect South Africans' higher overall trust in news sources. Our analysis further shows that South Africans relied more on content knowledge and contextual understanding when judging credibility, while participants from other countries emphasised formal linguistic features such as grammar and structure. Overall, the deviation from ideal rating was similar between groups (51% vs. 53%), suggesting that cultural familiarity appears to aid verification of authentic information but may also introduce bias when evaluating fabricated content. These insights contribute to understanding cross-cultural dimensions of misinformation detection and inform strategies for combating AI-generated fake news in increasingly globalised information ecosystems where content crosses cultural and geographical boundaries.

Problem

• Investigating cultural proximity effects on detecting AI-generated fake news
• Comparing human ability to identify fake versus true news across cultures
• Analyzing how cultural familiarity influences misinformation detection strategies

Innovation

• Survey comparing cultural groups detecting fake news
• Assessed reliance on content knowledge versus linguistic features
• Analyzed cultural familiarity impact on misinformation detection