🤖 AI Summary
This work addresses the challenges of context overflow, outdated state tracking, and escalating reasoning costs in enterprise workflows caused by verbose tool responses from large language model (LLM) agents. Focusing on a Microsoft Dynamics 365 expense reimbursement scenario, the authors propose a context management approach that integrates recent tool interaction pruning with automated summarization. Leveraging GPT-5 and Claude Sonnet 4.5 models via the Model Context Protocol, the method enables efficient context compression and multidimensional evaluation. Experimental results demonstrate that the approach achieves a 91.6% task completion rate and 99.64% monetary coverage using only 550k tokens and 5.79 hours of compute, significantly outperforming a full-history baseline while substantially reducing resource consumption without compromising performance.
📝 Abstract
Large language models deployed as autonomous agents for enterprise workflows face a key challenge: verbose tool responses from enterprise systems can cause context overflow, stale-state errors, and high inference cost. We study this problem in automated expense itemization in Microsoft Dynamics 365 Finance and Operations using Model Context Protocol tools. We evaluate four GPT-5 configurations on a 50-task hotel expense benchmark: no user model, full conversation history, context pruned to the last 5 tool call/response pairs, and pruning with automated summarization. Results are averaged across 5 independent runs, with the user model held constant for the context-engineering comparison. The no-user-model baseline achieves only 8.0% complete itemization. Full-context retention improves completion to 71.0%, but consumes 1,480,996 tokens and 14.56 hours per benchmark. Pruning to the last 5 tool calls improves completion to 79.0% while reducing token use to 535,274 and runtime to 5.39 hours. Adding summarization achieves the best result: 91.6% complete itemization and 99.64% average amount itemized, with 553,374 tokens and 5.79 hours. We further report confidence intervals, effect-size analysis, sensitivity over pruning and summary windows, failure analysis, results across five expense types grouped into three categories, and cross-model evidence with Claude Sonnet 4.5. These results show that, for this class of enterprise tool-use workflow, selective retention of recent tool interactions plus compact summarization can improve both reliability and efficiency compared with full-history retention.