Budgeted Dynamic Trace Structures for Token-Efficient Sequential Computation

2026-05-20

AI Summary

This work addresses the challenge of efficiently storing and querying long-sequence trajectory data—comprising nested branches, state transitions, and textual payloads—under strict token budget constraints. The paper proposes the Budgeted Dynamic Trajectory Structure (BDTS), a novel framework that unifies, for the first time, mechanisms including state-filtered reachability, cursor-based pagination, soft-bounded recent logs, reference-counted observation keys, incremental overwrites, bounded caching, and summary-suffix compression. BDTS maintains a rooted trajectory graph and an append-only history while rigorously preserving formal invariants within hard token limits. Experimental results demonstrate that BDTS achieves sub-3-millisecond construction and query latency on trajectories with tens of thousands of nodes, compressing raw inputs from 350K–2.71M tokens down to 1,048–4,120 tokens—reducing model input size by nearly 90%.

Abstract

Sequential computation increasingly produces long traces containing nested branches, status transitions, textual payloads, and compact summaries of earlier execution. This paper introduces budgeted dynamic trace structures (BDTS), a data-structural framework for maintaining rooted trace graphs and append-only histories under an explicit byte or token budget. BDTS combines status-filtered reachability, cursor pagination, soft-capped recency logs, reference-counted observation keys, delta overlays, bounded cost caches, and summary-plus-suffix compaction. We give formal invariants, asymptotic bounds, and an ancillary Rust implementation with reproducible benchmarks. Across synthetic traces with 10,000-40,000 vertices, the prototype builds graphs in 0.58-2.72 ms, enumerates all descendants in 0.24-1.42 ms, and compacts histories of 350k-2.71M approximate tokens to 1,048-4,120 approximate tokens. Tokenizer and forward measurements with three public model targets reduce 3,359-3,360 trace tokens to 432-433 tokens.

Problem

• sequential computation
• trace structures
• token budget
• long traces
• token efficiency

Innovation

• budgeted dynamic trace structures
• token-efficient computation
• trace compaction
• append-only history
• delta overlays