🤖 AI Summary
As the skill libraries of large language model (LLM) agents expand, operational costs escalate sharply, yet the effects of progressive disclosure strategies on retrieval quality and latency remain unclear. This study investigates enterprise-grade Workday agents to empirically quantify, for the first time, the performance trade-offs of lazy-loading-based progressive disclosure mechanisms in real-world production environments. Through systematic experimental evaluation, we demonstrate that this strategy significantly enhances skill retrieval quality while introducing only marginal response latency and effectively reducing operational costs. These findings provide critical empirical evidence for optimizing the architectural design of enterprise LLM agents, offering a practical pathway to balance scalability, efficiency, and system responsiveness in large-scale deployments.
📝 Abstract
Users of Workday's deployed LLM-based agents often request features which can be addressed by defining named procedures, also known as skills, in the LLM context, effectively augmenting agents'capabilities. However, as an agent's skills library grows in size, so does the agent's operational cost. Progressive disclosure (lazy-loading) of skills as needed may reduce operational costs, but its impact on overall latency and skill-retrieval quality remains unclear. In this report, we investigate the impact empirically and find that progressive disclosure improves skill-retrieval quality but marginally degrades overall latency.