An LLM-based Chain-of-Response Counter-Scam System

2026-05-31

AI Summary

This study addresses the challenge of rapidly evolving transnational cyber fraud and the lag in traditional detection and response systems by proposing an end-to-end, large language model (LLM)-based multi-agent anti-fraud framework. The framework introduces a security-oriented data specification tailored to fraud scenarios and establishes a Collaborative Semantic Response Task (CSRT) system comprising nine role-aligned natural language processing tasks, optimized through integration of diverse stakeholder requirements. It combines fraud-specific named entity recognition (NER), fine-tuned small LLMs (sLLMs), and a Collaborative Semantic Response Architecture (CSRA), leveraging a corpus of 185,000 real-world cases (CSRD). Experimental results demonstrate that the fine-tuned sLLMs outperform commercial models by over 10% across all CSRT tasks, while fraud NER achieves a 0.24 improvement in F1 score, significantly enhancing cross-organizational collaborative fraud response capabilities.

Abstract

The rapid evolution of online scams, driven by transnational networks and mass produced social engineering scenarios, has exposed the speed limitations of conventional detection, necessitating tighter interagency coordination. While LLMs show promise in scam identification, their role in accelerating integrated response frameworks remains underexplored. We propose Counter Scam, a unified LLM based multiagent framework that orchestrates end to end response from initial detection to crime investigation. The framework first proposes safe data guidelines, emphasizing nonpublic scam data and secure dataset construction via scam specific NER. Developed with insights from 37 stakeholders to reduce delays and improve analytical efficiency, the system integrates CSRA for multiagent mitigation, CSRT comprising nine role aligned NLP tasks, and CSRD, a corpus of 185,300 scam cases and 38,587 knowledge entries. Experiments show that fine tuned sLLMs surpass commercial models by more than 10% across all CSRT tasks and achieve a 0.24 F1 improvement in scam specific NER. These results demonstrate the framework's capability to enable rapid and collaborative mitigation of online scams.

Problem

• online scams
• response speed
• interagency coordination
• social engineering
• scam detection

Innovation

• LLM-based multiagent framework
• scam-specific NER
• role-aligned NLP tasks
• secure scam dataset construction
• collaborative counter-scam response