BrowserAgent: Building Web Agents with Human-Inspired Web Browsing Actions

2025-10-12

AI Summary

This work addresses the challenge of enabling large language models (LLMs) to autonomously interact with dynamic web environments—mimicking human behaviors such as clicking, scrolling, and typing—to retrieve information effectively. To this end, we propose BrowserAgent, an interactive agent framework that operates directly on real browsers via Playwright, bypassing static HTML-to-text conversion used by prior approaches (e.g., Search-R1, WebDancer). BrowserAgent incorporates an explicit memory mechanism to support long-horizon, multi-hop reasoning and employs a two-stage training strategy combining supervised fine-tuning (SFT) and rejection fine-tuning (RFT) to enhance generalization under low-data regimes. Evaluated on multi-hop question answering benchmarks—including HotpotQA, 2Wiki, and Bamboogle—BrowserAgent-7B achieves approximately 20% higher accuracy than Search-R1, demonstrating substantial gains over static-text-based paradigms.

Technology Category

• Multiagent Systems: Modeling other Agents
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Efficiently solving real-world problems with LLMs increasingly hinges on their ability to interact with dynamic web environments and autonomously acquire external information. While recent research like Search-R1 and WebDancer demonstrates strong performance in solving web tasks, they heavily rely on additional tools to convert the interactive web environment into static text content. This is in contrast to human browsing behaviors, which involve diverse interactions with the browser, such as scrolling, clicking, and typing. In this paper, we propose BrowserAgent, a more interactive agent that solves complex tasks through human-inspired browser actions. BrowserAgent operates directly on raw web pages via Playwright through a set of predefined browser actions. We adopt a two-stage training (Supervised Fine-Tuning (SFT) and Rejection Fine-Tuning (RFT)) to improve the model's generalization abilities. Despite using significantly less training data than Search-R1, BrowserAgent achieves more competitive results across different Open-QA tasks. Additionally, we introduce an explicit memory mechanism to store key conclusions across steps, further enhancing the model's reasoning capabilities for long-horizon tasks. Notably, BrowserAgent-7B can achieve around 20% improvement over Search-R1 on multi-hop QA tasks like HotpotQA, 2Wiki, and Bamboogle. These results indicate that BrowserAgent can serve as a more advanced framework for more interactive and scalable web agents.

Problem

• Building web agents with human-inspired browsing actions
• Solving complex tasks through interactive browser operations
• Enhancing reasoning for long-horizon web-based question answering

Innovation

• Human-inspired browser actions for web interaction
• Two-stage training enhances model generalization abilities
• Explicit memory mechanism improves reasoning for long tasks