Butterfly Effects in Toolchains: A Comprehensive Analysis of Failed Parameter Filling in LLM Tool-Agent Systems

2025-07-21

AI Summary

This work addresses the pervasive parameter-filling failure problem in large language model (LLM)-based tool-agent systems. We propose the first taxonomy of parameter failures specifically tailored to toolchains, categorizing them into five distinct failure modes and systematically analyzing their causal relationships with input sources. Through controlled experiments involving 15 types of input perturbations applied across mainstream tool invocation chains, we identify that naming hallucination stems primarily from intrinsic model limitations, whereas other failures are predominantly triggered by external factors—such as input noise and format deviations. Building on these findings, we introduce deployable mitigation strategies, including standardized return formatting and feedback-augmented inference mechanisms. Our empirical study provides foundational evidence for diagnosing reliability bottlenecks in tool agents and establishes a verifiable, technically grounded pathway toward enhancing parameter-filling robustness.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)
• Multiagent Systems: Other Foundations of Multi Agent Systems

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

The emergence of the tool agent paradigm has broadened the capability boundaries of the Large Language Model (LLM), enabling it to complete more complex tasks. However, the effectiveness of this paradigm is limited due to the issue of parameter failure during its execution. To explore this phenomenon and propose corresponding suggestions, we first construct a parameter failure taxonomy in this paper. We derive five failure categories from the invocation chain of a mainstream tool agent. Then, we explore the correlation between three different input sources and failure categories by applying 15 input perturbation methods to the input. Experimental results show that parameter name hallucination failure primarily stems from inherent LLM limitations, while issues with input sources mainly cause other failure patterns. To improve the reliability and effectiveness of tool-agent interactions, we propose corresponding improvement suggestions, including standardizing tool return formats, improving error feedback mechanisms, and ensuring parameter consistency.

Problem

• Analyzes parameter failure in LLM tool-agent systems
• Explores correlation between input sources and failures
• Proposes improvements for tool-agent interaction reliability

Innovation

• Construct parameter failure taxonomy
• Apply input perturbation methods
• Standardize tool return formats