JSPLIT: A Taxonomy-based Solution for Prompt Bloating in Model Context Protocol

2025-10-16

AI Summary

Large language model (LLM) agents suffer from prompt inflation as the number of available tools increases—leading to excessive prompt length, high token costs, elevated latency, and erroneous selection of irrelevant tools. Method: This paper proposes JSPLIT, a novel framework that integrates taxonomy-driven hierarchical tool categorization with semantic-aware dynamic tool filtering within the Model-Controller-Protocol (MCP) architecture, enabling real-time identification of the most relevant tool subset based on user queries. Contribution/Results: Evaluated on large-scale tool sets, JSPLIT significantly reduces prompt length (average 42% compression), lowers token consumption and response latency, and improves both tool selection accuracy and end-to-end task success rate—achieving simultaneous gains in efficiency and effectiveness.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Cognitive Modeling & Cognitive Systems: Agent ArchitecturesNatural Language Processing: Prompt Engineering / Prompting

Application Category

Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved informationSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactionsEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

AI systems are continually evolving and advancing, and user expectations are concurrently increasing, with a growing demand for interactions that go beyond simple text-based interaction with Large Language Models (LLMs). Today's applications often require LLMs to interact with external tools, marking a shift toward more complex agentic systems. To support this, standards such as the Model Context Protocol (MCP) have emerged, enabling agents to access tools by including a specification of the capabilities of each tool within the prompt. Although this approach expands what agents can do, it also introduces a growing problem: prompt bloating. As the number of tools increases, the prompts become longer, leading to high prompt token costs, increased latency, and reduced task success resulting from the selection of tools irrelevant to the prompt. To address this issue, we introduce JSPLIT, a taxonomy-driven framework designed to help agents manage prompt size more effectively when using large sets of MCP tools. JSPLIT organizes the tools into a hierarchical taxonomy and uses the user's prompt to identify and include only the most relevant tools, based on both the query and the taxonomy structure. In this paper, we describe the design of the taxonomy, the tool selection algorithm, and the dataset used to evaluate JSPLIT. Our results show that JSPLIT significantly reduces prompt size without significantly compromising the agent's ability to respond effectively. As the number of available tools for the agent grows substantially, JSPLIT even improves the tool selection accuracy of the agent, effectively reducing costs while simultaneously improving task success in high-complexity agent environments.

Problem

• Addressing prompt bloating in Model Context Protocol systems
• Reducing token costs and latency from excessive tool descriptions
• Improving tool selection accuracy for complex AI agents

Innovation

• Hierarchical taxonomy organizes MCP tools
• Query-based algorithm selects relevant tools
• Reduces prompt size while maintaining accuracy