AgentTime: Can Agents Estimate and Control Their Own Runtime?

2026-10-07

AI Summary

This study addresses the inability of AI agents to perceive time, precisely control execution duration, and predict task completion time by pioneering the exploration of native duration-following and control capabilities within agent frameworks. We construct AgentTime, a benchmark comprising 222 tasks spanning coding and computer-use scenarios, and conduct multi-round experiments with log analysis using frameworks such as Claude Code and Codex. Results reveal significant temporal deviation across models—varying up to 2.9-fold—with certain agents prematurely entering dormant states before task completion. Furthermore, removing temporal information substantially increases estimation errors. This work bridges the gap between temporal awareness and runtime control, providing critical evaluation criteria for enhancing the reliability of long-horizon autonomous agents.

Abstract

An essential control of AI agents is their ability to manage runtime. This ability requires a sense of time-awareness, to predict and estimate wall-clock time and to control their own actions. Prior work has focused on time-awareness, but duration-following and control in native agent harnesses remain unexplored. We present AgentTime, a benchmark for testing whether agents can work for a requested duration, predict their runtime, and estimate elapsed time afterward. It comprises 222 tasks from 18 sources spanning coding, computer use, agentic work, and automated research. Duration-following experiments append a single instruction specifying how long to work, with requests ranging from about a minute to multiple days. Accuracy on these instructions varies substantially: Fable 5.1 in Claude Code deviates from requested runtimes by a typical factor of 2.9$\times$, compared with only 1.2$\times$ for GPT-6 Astra in Codex. However, matching the requested runtime does not, by itself, establish continued work on the task. Among 158 reviewed Astra runs with classifiable transcripts, 14 explicitly slept after appearing to finish. In forecasting experiments, predictions tend to overestimate natural runtimes. In retrospective experiments, removing temporal information more than doubles deviation for Sol and Astra and nearly doubles it for Fable. An agent's ability to complete a task does not guarantee that it can control its own time or work for the whole requested duration. For agents to run reliably, safely, and autonomously over long horizons, we require the evaluation of both.

Problem

• AI agents
• runtime control
• time-awareness
• duration-following
• autonomous agents

Innovation

• AI Agents
• Runtime Estimation
• Time-awareness
• Duration Control
• Benchmark