You See It, They Don't: An Exploratory Study of User-to-User Variation in Instagram Comments

2026-03-23

AI Summary

This study presents the first empirical investigation into whether Instagram’s newly implemented AI-powered comment ranking system leads to divergent visibility of comments based on user attributes, with a particular focus on informational consistency in news-related content. By deploying four synthetic accounts varying in gender and political orientation and leveraging VPNs to simulate two geographic locations, the authors systematically collected visible comments under posts from 20 news and non-news accounts. Findings reveal that comment visibility across users exhibits significantly less variation for news posts compared to non-news posts. Moreover, account-level features—such as comment volume and follower count—account for more variance in ranking outcomes than user attributes like gender, political leaning, or location. These results underscore the dominant role of structural platform factors over individual user characteristics in shaping information visibility, offering new empirical insights into the mechanisms of personalization on social media.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Humans and AI: Intelligent User Interfaces
• Computer Vision: Bias, Fairness & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Social Networks and Social Media: Media and governance, opinion dynamics, filter bubbles, polarization
• Responsible Web: Algorithmic accountability and transparency on the web

Abstract

In March 2025, Meta announced a new AI system to rank the order of the comments shown to Instagram users. With existing research showing how feed personalization systems can lead to increased polarization, the introduction of this new system raises similar questions. This paper presents a small-scale exploratory study examining whether the ranking system produces systematic differences in visible comments shown to different users, particularly for news-related content. Using four sock-puppet accounts varying in gender and political leaning, we collect visible comments on posts from ten news and ten non-news accounts. This collection is repeated twice from two VPN locations to assess location effects. We ask 1) how many visible comments vary across different users, 2) is this variation higher for news accounts than non-news accounts, and 3) can user-attributes like gender, political leaning, and location systematically explain the observed variation. Contrary to our expectations, we find that visible comments on news posts are less likely to vary across users than those on non-news posts. Variation is better explained by account metrics like comment and follower counts than by user attributes. These findings provide an initial glimpse into personalized comment ranking on Instagram and motivate larger, more systematic audits of how comment personalization may shape online discourse. To support further research, we provide the code to collect comments and the data upon request.

Problem

• comment personalization
• user variation
• Instagram
• news content
• algorithmic ranking

Innovation

• comment personalization
• algorithmic auditing
• social media polarization
• sock-puppet accounts
• Instagram ranking system