ScamPilot: Simulating Conversations with LLMs to Protect Against Online Scams

2026-01-30

AI Summary

This study addresses the growing challenge of evolving online scams, which outpace existing automated defense systems in equipping users to recognize novel fraud tactics. To bridge this gap, the authors propose a conversational anti-fraud training framework powered by large language models, featuring two interacting agents—one simulating a scammer and the other a potential victim—to dynamically recreate realistic scam scenarios. The approach integrates real-time user intervention with multiple-choice prompts, encouraging participants to provide actionable advice that reinforces fraud awareness. In a controlled experiment involving 150 participants, the method significantly improved scam identification accuracy by 8%, response effectiveness by 9%, and self-efficacy by 19%. Notably, users predominantly offered action-oriented recommendations without compromising trust in legitimate interactions.

Technology Category

• Application Domains: Misinformation & Fake News
• Multiagent Systems: Adversarial Agents
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Mitigating online harms, including harassment, spam, and fake reviews
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Fraud continues to proliferate online, from phishing and ransomware to impersonation scams. Yet automated prevention approaches adapt slowly and may not reliably protect users from falling prey to new scams. To better combat online scams, we developed ScamPilot, a conversational interface that inoculates users against scams through simulation, dynamic interaction, and real-time feedback. ScamPilot simulates scams with two large language model-powered agents: a scammer and a target. Users must help the target defend against the scammer by providing real-time advice. Through a between-subjects study (N=150) with one control and three experimental conditions, we find that blending advice-giving with multiple choice questions significantly increased scam recognition (+8%) without decreasing wariness towards legitimate conversations. Users'response efficacy and change in self-efficacy was also 9% and 19% higher, respectively. Qualitatively, we find that users more frequently provided action-oriented advice over urging caution or providing emotional support. Overall, ScamPilot demonstrates the potential for inter-agent conversational user interfaces to augment learning.

Problem

• online scams
• phishing
• impersonation scams
• scam recognition
• user protection

Innovation

• conversational simulation
• large language models
• scam prevention
• inter-agent interface
• user inoculation