Rethinking Inference-Time Scaling in Local Computer-Use Agents: Failure Modes and Compute Tradeoffs

2026-07-30

AI Summary

This study addresses the performance bottlenecks of resource-constrained local agents under stringent hardware limitations by systematically investigating four dimensions of inference-time scaling: context, time, structure, and parallelism. Experiments on the OSWorld benchmark using Qwen3-VL-8B/30B-A3B, UI-TARS-1.5-7B, and OpenCUA-7B models reveal that context extension enhances trajectory stability but exhibits diminishing returns, temporal extension alleviates stuttering yet yields limited gains in task success rates, and parallel execution reduces structural overhead at the cost of high computational expense. The work uncovers, for the first time, the law of diminishing returns in inference scaling and a shift in failure modes, leading to a novel paradigm that integrates selective computation allocation with failure-aware control—offering both theoretical grounding and practical pathways for designing efficient local agents.

Abstract

Deploying autonomous computer-use agents (CUAs) locally is increasingly important for privacy, cost efficiency, and practical usability, yet improving their performance under strict hardware constraints remains challenging. While recent studies show that inference-time scaling can improve frontier computer-use agents through additional computation during execution, its effectiveness for resource-constrained local models remains poorly understood. We present a systematic empirical study of inference-time scaling in local CUAs across contextual, temporal, structural, and parallel dimensions. We evaluate Qwen3-VL-8B/30B-A3B, UI-TARS-1.5-7B, and OpenCUA-7B on the OSWorld benchmark. Our results show that additional computation often yields diminishing returns while changing failure modes. Contextual scaling provides historical grounding that improves trajectory stability and task accuracy, but its gains saturate as token cost increases and failures shift from repetitive or stalled trajectories toward premature false successes. Temporal scaling similarly reduces max-step stalls, yet does not substantially improve task success, indicating that longer horizons often extend erroneous trajectories rather than correct them. We further find that structural decomposition can introduce planning and formatting overhead in local two-stage agents, while parallel scaling partially mitigates these failures at a substantial computational cost. Overall, our findings suggest that efficient local CUAs require selective compute allocation, failure-aware control mechanisms, and agentic frameworks designed around the capabilities and limitations of local models.

Problem

• computer-use agents
• inference-time scaling
• local deployment
• failure modes
• compute tradeoffs

Innovation

• inference-time scaling
• local computer-use agents
• failure modes
• compute tradeoffs
• agent efficiency