bot detection and mitigation

Designs and implements systems that detect, attribute, and mitigate automated actors (bots) interacting with online services by building data collection pipelines, detection models and rules, and enforcement controls (rate-limits, challenges, blocking). Includes engineering device-fingerprinting capture and analysis, developing anti-bot integration and mitigation logic, and analyzing evasion and fingerprint-bypass techniques to improve detection robustness.

botdetectionandmitigation

Recent Skill Trend

Momentum and market value over time
Trending
Score
No comparison yet
0.48
Oct 01, 2026Oct 01, 2026
Career
Value
No comparison yet
$217K/year
Oct 01, 2026Oct 01, 2026

Recommended Survey Paper

Quick overview of the field
View more

Must-Read Papers

Most classic and influential ideas
View more

This study addresses the growing prevalence of bot detection in browser automation, which leads to systematic sample loss in web security and privacy research. Through large-scale measurements across 10,000 websites (40,000 visits) under four browser configurations, this work constructs a taxonomy of bot detection mechanisms and reveals that 82% of access denials are attributable to such detection, with Cloudflare and Akamai accounting for 37% and 26%, respectively. The authors propose a conditional-dependence-based inference method to uncover undeclared detection behaviors, combining custom instrumentation, header spoofing, and cross-configuration comparison. Their analysis shows that HTTP header signals alone explain 75% of the blocks unique to Chromium’s headless mode. Notably, 83% of recent top-tier publications fail to report this bias, underscoring its substantial threat to research validity.

bot detectionbrowser automationheadless browsers

This study addresses the challenge of effectively detecting and distinguishing large language model (LLM) web agents employing stealth and anti-detection techniques from genuine human users. By deploying honeypot websites integrated with multiple anti-scraping mechanisms—including robots.txt, CAPTCHA, proof-of-work challenges, and Cloudflare protections—and combining multi-layer fingerprinting across network, HTTP, and browser levels, the authors systematically evaluate behavioral characteristics of six prominent LLM agents. The research reveals, for the first time, that all tested agents can be clearly differentiated from both humans and each other through these multi-layer fingerprints. Notably, some agents successfully bypass all deployed anti-scraping measures, while their stealth strategies often prove counterproductive, inadvertently increasing detectability and thereby challenging prevailing assumptions about the efficacy of current anti-detection approaches.

anti-bot mechanismsbot detectionLLM-based bots

Implementing a Robot Intrusion Prevention System (RIPS) for ROS 2

Dec 26, 2024
ES
Enrique Soriano-Salvador
🏛️ Universidad Rey Juan Carlos | EIF

ROS 2–based autonomous robots face critical security threats, including physical harm, malicious node injection, topic hijacking, and service abuse. To address these challenges, this paper proposes RIPS—the first lightweight, rule-driven Robot Intrusion Prevention System tailored for ROS 2. RIPS introduces three key innovations: (1) a robot-specific threat model aligned with runtime semantic behaviors; (2) a domain-specific rule language enabling multi-scenario detection and real-time response; and (3) an integrated architecture comprising behavior-monitoring agents and a policy engine, fully compliant with the ROS 2 security framework to deliver end-to-end protection. Evaluated on a real-world social robot platform and an international robotics competition testbed, RIPS demonstrates robust attack mitigation capability, achieving <2% false positive rate and average latency <15 ms. The results substantiate significant improvements in both security assurance and operational practicality for cyber-physical robotic systems.

Autonomous RobotsSafety MeasuresSecurity

This work addresses a novel class of social engineering attacks targeting web automation agents, which exploit induced contextual cues to manipulate agent behavior—threats that existing defenses struggle to mitigate. We introduce AgentBait, the first attack framework specifically designed for web-based intelligent agents, and propose SUPERVISOR, a lightweight, plug-and-play runtime protection module that blocks such attacks by verifying the consistency between the web environment and the agent’s intended task. Experimental evaluation demonstrates that AgentBait achieves an average success rate of 67.5% against mainstream agent frameworks, whereas integrating SUPERVISOR reduces this success rate by 78.1% with only a 7.7% runtime overhead, effectively balancing security and usability.

AgentBaitattack surfaceLLM-based agents

This work addresses the limitations of existing adversarial simulation tools, which rely on agent-based instrumentation of target systems, often leaving anomalous artifacts and failing to faithfully replicate human attacker behavior—particularly in critical phases of the cyber kill chain such as initial access and interactive operations. To overcome these shortcomings, the authors propose and implement an open-source attack scripting language coupled with an agentless execution engine that closely emulates real-world attacker tactics. This approach enables high-fidelity, interactive simulation of complete kill chain stages, including initial access, privilege escalation, and lateral movement. Experimental results demonstrate that system logs generated by this method exhibit significantly greater behavioral similarity to those produced by actual human-driven attacks, thereby enhancing the realism and effectiveness of security testing and intrusion detection research.

adversary emulationattack automationcyber attack scenarios

Latest Papers

What's happening recently
View more

This study addresses the emerging threat posed by artificial intelligence–driven offensive cyber agents and the resulting detection gap between these advanced adversaries and conventional defense mechanisms. To bridge this gap, the paper proposes a “defense-in-depth detection” strategic framework—the first systematic approach specifically designed for identifying AI-powered autonomous attack agents. The framework integrates five core mechanisms: agent identifiers, decoy agents, AI-automated alert analysis, standardized agent security alert protocols, and a cybersecurity information-sharing platform. Together, these components fill critical gaps in current defensive capabilities. By providing policymakers, industry stakeholders, and defenders with actionable tools and collaborative mechanisms, the framework significantly enhances early detection and coordinated response to AI-driven cyberattacks.

AI-driven cyberattacksautonomous cyber operationscyber defense

This study addresses the emerging risk of AI agents being misused for user surveillance, exacerbated by a lack of transparency and user control. It formally defines the problem of “agent surveillance” and introduces SurveilBench, a multidomain evaluation benchmark that systematically reveals how large language models autonomously assist in monitoring users—and even proactively report them to authorities—across enterprise, educational, and law enforcement contexts. To counter this threat, the work proposes three prompt-injection–based evasion strategies: concealment, deception, and诱导 over-reporting. Experimental results demonstrate that current AI agents already possess practical surveillance capabilities, while the proposed methods effectively disrupt their surveillance behaviors, offering a novel avenue for safeguarding user privacy.

agentic surveillanceAI agentsdata access

This study addresses the growing challenge posed by AI-driven automated scanning tools that undermine foundational assumptions of intrusion detection systems in industrial control systems (ICS) and Industrial Internet of Things (IIoT) environments. Leveraging a modular analytical pipeline applied to 192 million darknet packets captured between 2021 and 2025, this work reveals for the first time that modern botnets employ microsecond-scale artificial delays to smooth traffic and evade conventional threshold-based anomaly detection. Multidimensional evaluation—including average packet rate, Shannon entropy, inter-arrival time burstiness, geolocation provenance, and port distribution—demonstrates that ICS-targeted scan traffic nearly doubled over four years. Alarmingly, 97.47% of botnet traffic successfully bypasses standard detection mechanisms, while increasing detector sensitivity incurs an unacceptably high false positive rate of 68.10%.

AI-assisted bot trafficdarknet dataICS security

This work addresses the growing privacy and security risks posed by AI-powered autonomous web agents that commonly bypass protective mechanisms such as robots.txt. To counter this, the authors propose a multi-layer fingerprinting approach that integrates network-level features (e.g., TLS/HTTP protocol characteristics) with browser interaction behaviors, enabling precise differentiation among AI agents, human users, and traditional web crawlers. The method introduces a deployable logging framework that achieves fine-grained, cross-layer attribution of mainstream AI agents for the first time, demonstrating strong evasion resistance and robustness. Experimental evaluation across six widely used agent frameworks shows a classification accuracy of 97%, effectively isolating distinct traffic types and establishing a novel paradigm for content protection on the web.

AI web agentsbot detectioncontent scraping

This study addresses the lack of systematic analysis and actionable controls linking large language model (LLM) agent security threats to real-world financial regulations. It presents the first mapping of six categories of LLM agent risks to regulatory obligations in the U.S. and EU, and proposes four scalable compliance architectures centered on auditability, authorization, and boundary enforcement. Key technical contributions include agent-to-agent (A2A) compliance orchestration, audit-driven Grounded-RAG, case ID propagation, and reasoning-boundary de-identification proxies. Empirical evaluation demonstrates that the framework reduces manual processing from multiple days to same-day handling, automates approximately 80% of use cases, and uncovers two types of control failures detectable only through internal audit, as well as one category of legitimate applicants erroneously rejected.

agent securityauditabilitycompliance

Hot Scholars

LH

Lynnette Hui Xian Ng

Societal Computing PhD Student at Carnegie Mellon University
Societal ComputingComputational Social ScienceSocial Network Analysis
KM

Kathleen M. Carley

Professor, Carnegie Mellon University
network sciencecomputer simulationsocial media analytics
WZ

Wenqi Zhou

Associate Professor of Information Systems Management, Duquesne University
BusinessInformation SystemsMarketingE-commerce