Chatbot Engagement Does Not Always Beget Metalearning: Evidence from Three Countries

2026-09-26

AI Summary

This study investigates whether chatbot-mediated error correction can produce durable meta-learning effects rather than merely rectifying erroneous beliefs instantaneously. To this end, we conducted a preregistered randomized controlled trial across three countries, deploying a multi-channel intervention utilizing a Socratic chatbot grounded in verification detectors to evaluate its immediate and one-week retention effects on visual misinformation discernment and sharing intentions. This work makes the first empirical distinction between immediate corrective impacts and sustained meta-learning outcomes. Results indicate that while the chatbot significantly enhanced immediate discernment and suppressed false information sharing, these advantages dissipated after one week. These findings reveal a critical insight: high-engagement interactions cannot substitute for the gradual development of AI literacy.

Abstract

Chatbots deliver real-time fact-checks, but whether a chatbot correction leaves anything behind once the chatbot is gone - metalearning, distinct from correcting misbeliefs - is untested. We report a preregistered, three-country randomized experiment (USA, India, Singapore; N ~ 2,200) on out-of-context image misinformation, manipulating a correction's channel affordances (synchronicity, bandwidth) across four conditions: Control, Links-only, Static explanation, and a Socratic Chatbot built on a validated out-of-context detector, with an unaided retest one week later. The Chatbot produced the largest immediate discernment gain (d = 0.097, p = .023). All three interventions reduced sharing of false claims (d ~ -0.12, p<.01). One week later, no advantage persisted: the Chatbot arm declined relative to Control, most sharply in India and Singapore, and in India on claims it never discussed. Decay tracked affordance level and did not vary by country. Engagement mechanisms, we argue, do not substitute for slow AI literacy.

Problem

• chatbot engagement
• metalearning
• misinformation
• fact-checking
• AI literacy

Innovation

• Socratic Chatbot
• Out-of-context Image Misinformation
• Channel Affordances
• Metalearning
• Randomized Controlled Trial