Rethinking AI Cloud Infrastructure for Agentic Serving Systems with the Aries Experimentation Framework

2026-07-31

AI Summary

Current large model inference architectures struggle to efficiently support agent-centric requirements such as persistent context maintenance, repeated reasoning, and secure, sandboxed tool invocation. This work proposes Aries—a novel agent-native serving paradigm that decouples task semantics from execution configuration, unifies heterogeneous sandboxed tool interfaces through stateful abstractions, and integrates trajectory reconstruction with system telemetry to enable a reproducible, full-stack experimental framework. Key contributions include introducing trajectory-level performance metrics that expose how token-based metrics overlook non-inference bottlenecks; demonstrating diminishing returns from extended context alongside its negative impact on service capacity; observing that sandboxes exhibit idle-burst usage patterns rendering snapshot-based state management costly; and leveraging these insights to design adaptive context management, elastic sandbox scheduling, and a lightweight sandbox mechanism with minimal attack surface.

Abstract

Autonomous agents challenge conventional LLM serving by coupling repeated inference with persistent context and sandboxed tool execution. We present Aries, a full-stack experimentation framework that separates task semantics from execution configurations, reconstructs cross-component agent trajectories with correlated system telemetry, and exposes stateful tool execution through a consistent interface across heterogeneous sandbox substrates. We use Aries to conduct reproducible experiments on open agent harnesses and benchmarks. We complement these experiments with production traces from a commercial platform, grounding low-level systems research in observed production behavior. Our results show that (1) token-centric metrics miss non-inference bottlenecks, (2) retaining additional context yields diminishing accuracy benefits while reducing serving capacity, and (3) tool sandboxes alternate between long idle periods and short resource bursts, while current snapshot-based state management makes aggressive suspension costly. A complementary security analysis further highlights the need to reduce the sandbox attack surface. We then discuss the vision for agent-native serving systems designed around trajectory-level metrics, adaptive context management, elastic sandbox resource management, and sandboxes with minimized attack surface.

Problem

• agentic serving
• LLM inference
• tool sandboxing
• context management
• AI infrastructure

Innovation

• agentic serving
• experimentation framework
• stateful tool execution
• sandbox resource management
• trajectory-level telemetry