🤖 AI Summary
This study addresses the issues of residual stale outputs and loss of valid state during task revision in LLM agents by proposing a versioned execution control plane. The proposed method transforms abort-and-restart operations into seamless version transitions, decouples resource scheduling from release permissions, and unifies fast-fail with selective retention mechanisms to support the inheritance of completed KV cache states. A vLLM-based system implementation encompasses GPU execution, hierarchical recovery, and multi-tenant serving. Experimental results demonstrate that this approach reduces the median time-to-first-token latency by 17.1% while eliminating obsolete outputs and effectively preserving execution progress across repeated revisions.
📝 Abstract
LLM agents revise running tasks when users change instructions, tools fail, or new information changes a plan. Today's servers express a revision as aborting old requests and submitting replacements. Yet the old execution's buffered output and outstanding work must stop affecting the application, while completed KV state may still be useful to its replacement. Handling these obligations separately can leave obsolete effects publishable and force the successor to rebuild valid state.
We present \retire, a serving control-plane redesign around \emph{versioned execution}. Requests own scheduling and memory resources; execution versions own \emph{authority}, the permission to publish output or install state for the current execution. \retire first revokes obsolete work, then bounds its remaining execution and certifies the completed prefix its successor can inherit. The successor runs from that state while isolated old resources are reclaimed asynchronously. This unifies fast invalidation and selective preservation in one version transition.
We implement \retire in vLLM across output publication, GPU execution, KV handoff, tiered recovery, and distributed and multi-tenant serving. Correctness experiments verify current-version output and valid state inheritance across these paths. Combining invalidation with inheritance reduces revision-to-successor time-to-first-token by a median 17.1\% in controlled paired experiments. A replay of recorded coding-agent interruption arrivals emits no obsolete output and keeps every final version progressing through repeated revisions. \retire turns abort-and-restart into a coordinated handoff that stops obsolete work quickly and preserves useful work for its successor.