Fact-Check Your Information (FYI): A Design Probe to Understand How People Actually Fact-Check Data-Driven Articles

2026-08-07

AI Summary

This work addresses the disconnect between automated and manual verification in existing fact-checking systems, which hinders readers’ ability to flexibly collaborate with verification tools during reading. The authors propose FYI, a browser extension that embeds fact-checking directly into the reading environment and offers four complementary tools spanning fully automated to manually driven exploration. FYI establishes a collaborative verification paradigm centered on visualization, positioning AI as a starting point rather than an authoritative source. An exploratory study with 22 participants revealed three prevalent user workflows—AI-first, manual-first, and parallel co-auditing—and demonstrated that visualization serves as a critical mechanism for auditing AI-generated claims. Trust in the system increased when multiple tools converged on consistent results and decreased when they diverged. The system is open-sourced to advance research in hybrid verification.

Abstract

Data-driven journalism and policy reports frequently rely on statements grounded in statistical evidence, referred to as data claims. Verifying such a claim requires connecting it to the underlying structured dataset. However, existing systems typically isolate automated fact-checking from manual data exploration, leaving it unclear how readers coordinate AI assistance with manual inspection of the evidence in practice. We present FYI, a browser extension that embeds fact-checking in the reading environment, and use it as a design probe to study how people detect, verify, and determine the validity of data claims against the underlying dataset. FYI provides four complementary tools spanning the spectrum from full automation to manual data exploration. In an exploratory study (N=22), participants used FYI to fact-check claims in a data-driven article. We find that participants adopted three distinct workflow archetypes---AI-first with manual confirmation, manual-first with AI supplement, and parallel co-review---with visualization serving as the primary mechanism for auditing AI conclusions. Trust in AI shifted dynamically, growing when multiple tools converged and eroding when AI outputs were inconsistent. These findings suggest that fact-checking systems should treat AI as a starting point that human verification complements rather than a definitive authority, elevate visualization as a core verification capability, and support flexible, user-driven workflows. We release FYI as open-source software for further research at https://github.com/DataVisards/FYI.

Problem

• fact-checking
• data claims
• AI assistance
• manual verification
• data-driven journalism

Innovation

• fact-checking
• AI-assisted verification
• data-driven journalism
• interactive visualization
• user workflow