"It Felt Real" Victim Perspectives on Platform Design and Longer-Running Scams

2025-10-02

AI Summary

This study investigates how digital platform design features are exploited to perpetuate long-term online scams—particularly romance scams and “pig-butchering” fraud—and amplify their persistence and harm. Drawing on in-depth interviews with 25 Chinese victims and thematic analysis, it identifies, for the first time from human-computer interaction and socio-technical systems perspectives, four high-risk design mechanisms: (1) inadequate identity verification enabling perpetrator impersonation; (2) algorithmic recommendation systems that increase victim-perpetrator contact; (3) private messaging interfaces fostering social isolation; and (4) integrated payment systems facilitating illicit fund transfers. Moving beyond conventional user-centric anti-fraud paradigms, the study proposes “anti-scam design” as a novel intervention strategy targeting platform architectures themselves. It contributes empirically grounded insights and actionable design principles to inform platform safety governance and responsible technology development.

Technology Category

• Multiagent Systems: Mechanism Design
• Application Domains: Misinformation & Fake News
• Game Theory and Economic Paradigms: Mechanism Design

Application Category

• Responsible Web: Mitigating online harms, including harassment, spam, and fake reviews
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Longer-running scams, such as romance fraud and "pig-butchering" scams, exploit not only victims' emotions but also the design of digital platforms. Scammers commonly leverage features such as professional-looking profile verification, algorithmic recommendations that reinforce contact, integrated payment systems, and private chat affordances to gradually establish trust and dependency with victims. Prior work in HCI and criminology has examined online scams through the lenses of detection mechanisms, threat modeling, and user-level vulnerabilities. However, less attention has been paid to how platform design itself enables longer-running scams. To address this gap, we conducted in-depth interviews with 25 longer-running scam victims in China. Our findings show how scammers strategically use platform affordances to stage credibility, orchestrate intimacy, and sustain coercion with victims. By analyzing scams as socio-technical projects, we highlight how platform design can be exploited in longer-running scams, and point to redesigning future platforms to better protect users.

Problem

• Examining how platform design enables longer-running scams
• Analyzing scammers' strategic use of platform affordances
• Highlighting need for platform redesign to protect users

Innovation

• Analyzing scams as socio-technical projects
• Examining platform design enabling longer scams
• Redesigning future platforms for user protection