🤖 AI Summary
Existing real-time web agents evaluate task completion solely based on the final answer, neglecting implicit strategic constraints in user instructions—such as verification requirements, preferences, and termination conditions—resulting in poor compliance despite high completion rates. This work proposes WebRider, an architecture that models browsing trajectories as first-class objects and formalizes user intent through “intent contracts.” It employs hierarchical control: a top layer preserves contract semantics, a middle layer generates constrained actions, and a bottom layer integrates browser, search, and mapping tools. We introduce RiderBench, a benchmark comprising 4,096 real-time contracts, and train an 8B-parameter model. Experiments reveal that while strong controllers achieve a 99.2% task completion rate, only 38.8% fully satisfy all constraints; WebRider significantly outperforms baselines relying solely on executable actions under a fixed controller, demonstrating the critical role of contract-based mechanisms in ensuring strategic compliance.
📝 Abstract
Delegating a web task involves more than asking a question; it requires transferring a policy: what to verify, how to handle uncertainty, which preferences matter, and when to stop. Yet, current live-web agents are evaluated solely on the final answer, ignoring the policy constraints that define the delegation. A plausible final answer can conceal violations of that policy. Our full live audit reveals this critical gap: a strong controller completes 99.2% of tasks but honors all policy constraints in only 38.8% of cases. Finishing does not imply fidelity. WebRider bridges this gap by formalizing the delegated policy as an intent contract---an operational record of goals, constraints, evidence obligations, answer form, and task-local persona controls that must hold even as web pages change. WebRider employs a hierarchical architecture: a top-layer controller maintains the contract, a middle layer realizes intentions as guarded executable actions, and a tool layer executes these actions via browser, search, and maps tools. Our benchmark, RiderBench, evaluates this design on 4,096 live-web contracts across 42 public websites, auditing both the internal contract state and the visible user experience to determine if a rollout preserved its policy and if the steps were persona-consistent. The guarded middle interface also serves as a high-quality training signal; an 8B action-policy model trained through this interface outperforms executable-only baselines under a fixed controller. By making the browsing path a first-class object, WebRider enables a system that is auditable, human-judgeable, and learnable without conflating action realization with final-answer decisions. Dataset URL: hf.co/datasets/WebRider/WebRider.