AttackPilot: Autonomous Inference Attacks Against ML Services With LLM-Based Agents

2025-11-24

AI Summary

Non-experts face significant challenges in executing inference attacks and tuning parameters to assess security risks of machine learning (ML) services. Method: This paper proposes a large language model (LLM)-based multi-agent automated evaluation framework. It introduces a task-customized action space and an adaptive policy optimization mechanism to mitigate common agent failures—such as planning errors, instruction deviation, and context loss—and employs models like GPT-4o to instantiate autonomous attack agents that dynamically adapt behaviors under constrained API environments. Contribution/Results: Evaluated on 20 real-world ML services, the framework achieves a 100% task completion rate with an average cost of only $0.627 per attack, matching human expert performance in efficacy. To our knowledge, this is the first work enabling low-barrier, highly robust, near-expert-level automation for inference attack evaluation.

Technology Category

• Multiagent Systems: Adversarial Agents
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Inference attacks have been widely studied and offer a systematic risk assessment of ML services; however, their implementation and the attack parameters for optimal estimation are challenging for non-experts. The emergence of advanced large language models presents a promising yet largely unexplored opportunity to develop autonomous agents as inference attack experts, helping address this challenge. In this paper, we propose AttackPilot, an autonomous agent capable of independently conducting inference attacks without human intervention. We evaluate it on 20 target services. The evaluation shows that our agent, using GPT-4o, achieves a 100.0% task completion rate and near-expert attack performance, with an average token cost of only $0.627 per run. The agent can also be powered by many other representative LLMs and can adaptively optimize its strategy under service constraints. We further perform trace analysis, demonstrating that design choices, such as a multi-agent framework and task-specific action spaces, effectively mitigate errors such as bad plans, inability to follow instructions, task context loss, and hallucinations. We anticipate that such agents could empower non-expert ML service providers, auditors, or regulators to systematically assess the risks of ML services without requiring deep domain expertise.

Problem

• Automating inference attacks on ML services for non-experts using LLM-based agents
• Achieving expert-level attack performance autonomously without human intervention
• Enabling systematic ML risk assessment without requiring deep domain expertise

Innovation

• Autonomous agent conducts inference attacks automatically
• Multi-agent framework mitigates planning and execution errors
• LLM-powered agent adapts strategies under service constraints