Revising Context, Shifting Simulated Stance: Auditing LLM-Based Stance Simulation in Online Discussions

2026-06-04

AI Summary

This study investigates whether large language models (LLMs) genuinely reflect authentic user beliefs when simulating stances or are instead overly sensitive to minor contextual perturbations. To this end, the authors propose the first counterfactual context revision framework that integrates textual and multimodal cues—such as emojis—to systematically evaluate the robustness of LLM-based stance simulation. By incorporating a polarization preference mechanism, the framework quantifies both the direction and rate of stance shifts under controlled variations. Experimental results across multiple polarization settings reveal that LLMs consistently exhibit significant and predictable stance changes, demonstrating that their simulated positions are highly controllable yet fragile. These findings underscore the substantial influence of contextual manipulation on LLM-generated stances.

Abstract

Large language models are increasingly used to simulate social media users and infer how individuals may respond to online discussions. However, it remains unclear whether these simulations reflect precise user-specific beliefs or whether they are highly sensitive to semantically independent changes in conversational contexts. In this work, we study counterfactual context revision as a framework for auditing LLM-based stance simulation. Given an original online conversation, we first infer a target user's stance toward a specific topic. We then apply controlled revision strategies to the conversational context and simulate the user's stance again under the revised context. We compare text-only revision strategies with a multimodal one that incorporates meme-based context and evaluate two main effectiveness metrics, i.e., average directional stance shift and stance transition rate. The results reveal effective and robust stance transitions in both text-only and multimodal strategies across different polarization-preference mechanisms. Our study contributes an evaluation framework for understanding the context sensitivity of LLM-based stance simulation. More broadly, it highlights both the promise and risk of using LLMs to simulate online opinion dynamics.

Problem

• stance simulation
• context sensitivity
• large language models
• online discussions
• counterfactual revision

Innovation

• counterfactual context revision
• LLM-based stance simulation
• context sensitivity
• multimodal context
• stance transition