Iffy-Or-Not: Extending the Web to Support the Critical Evaluation of Fallacious Texts

2025-03-18

AI Summary

Social media comments frequently contain unverified or false information, undermining users’ formation of informed opinions. To address this, we propose Iffy-Or-Not (ION), a lightweight browser extension that—uniquely among real-time reading interventions—integrates the three core elements of argumentation theory: fallacy detection, question-guided prompting, and collaborative reflection. ION dynamically highlights logical fallacies during browsing using heuristic rules and generates Socratic-style questions to stimulate critical thinking. A mixed-methods evaluation (N=18) demonstrates that ION significantly enhances textual attention and the quality of user-generated critiques; 87% of participants rated ION’s questions as superior to their self-formulated ones. Crucially, the study reveals that misaligned targeting—e.g., over-intervention on low-risk content—can trigger reactance, thereby reducing efficacy. These findings provide empirically grounded design principles and methodological innovations for trustworthy, AI-augmented cognitive interventions.

Technology Category

• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Application Domains: Humanities & Computational Social Science
• Knowledge Representation and Reasoning: Argumentation

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Social platforms have expanded opportunities for deliberation with the comments being used to inform one's opinion. However, using such information to form opinions is challenged by unsubstantiated or false content. To enhance the quality of opinion formation and potentially confer resistance to misinformation, we developed Iffy-Or-Not (ION), a browser extension that seeks to invoke critical thinking when reading texts. With three features guided by argumentation theory, ION highlights fallacious content, suggests diverse queries to probe them with, and offers deeper questions to consider and chat with others about. From a user study (N=18), we found that ION encourages users to be more attentive to the content, suggests queries that align with or are preferable to their own, and poses thought-provoking questions that expands their perspectives. However, some participants expressed aversion to ION due to misalignments with their information goals and thinking predispositions. Potential backfiring effects with ION are discussed.

Problem

• Enhance critical thinking in evaluating online texts
• Identify and highlight fallacious content in web articles
• Provide tools to resist misinformation on social platforms

Innovation

• Browser extension for critical text evaluation
• Highlights fallacies and suggests probing queries
• Encourages critical thinking and perspective expansion