LIBERO-Agent: Evaluating General-Purpose Agents for Direct Embodied Manipulation

2026-09-30

AI Summary

This study addresses the unclear reliability of transferring general-purpose agent capabilities to embodied manipulation. To this end, we propose a native action-instruction interaction framework and construct a unified benchmark integrating 200 tasks with tool-calling mechanisms. This benchmark systematically evaluates agents’ perception, execution, and long-horizon compositional generalization in robotic manipulation, enabling fine-grained differentiation across multiple difficulty levels. Our investigation reveals significant performance degradation patterns among agents in complex manipulation tasks. Experimental results demonstrate that GPT-6 Astra achieves the best overall performance yet still exhibits geometric reasoning errors. These findings provide critical empirical evidence regarding the capability boundaries of embodied agents.

Abstract

General-purpose agents can plan, use tools, and revise their behavior from feedback, but it remains unclear whether these capabilities transfer from digital environments to embodied manipulation. To investigate this question, we introduce LIBERO-Agent, an agent-native benchmark for evaluating these agents in robot manipulation tasks. Rather than asking agents to submit task-level Python control programs or operate through high-level robot skills, LIBERO-Agent provides an interactive robotic environment where agents can select which observations to inspect, process them with their own tools, and issue native action commands. LIBERO-Agent integrates 200 tasks into a common interaction framework and provides a 30-task primary suite that separates perception, short-horizon execution, and long-horizon composition. Results reveal a pronounced reliability gap: while agents perform well on perception and easy short-horizon tasks, their performance degrades substantially on hard short-horizon and long-horizon tasks. Richer observations improve short-horizon manipulation, while demonstration benefits depend on the agent and format. Among these agents, GPT-6 Astra achieves the strongest overall performance. Further analysis shows its major advantage lies in mechanism interaction, especially when sustained physical contact is needed, while its remaining failures stem from cross-stage interference and geometric errors.

Problem

• General-purpose agents
• Embodied manipulation
• Robot manipulation
• Benchmark evaluation
• Reliability gap

Innovation

• Agent-native benchmark
• Embodied manipulation
• Interactive robotic environment
• Native action commands
• General-purpose agents