In-Context Learning for Robots: Methods and Applications

2026-09-28

AI Summary

This study addresses the challenge of enabling general-purpose robots to infer novel tasks from demonstrations and translate them into physical actions under fixed parameters. It presents a systematic review of in-context learning for robotics, categorizing interaction interfaces into four types and examining the roles of training, correspondence, and memory mechanisms in manipulation and navigation. The work synthesizes techniques including context-conditioned policies, geometric demonstration transfer, world-model-based control, and skill agents. Its core contributions lie in clarifying transfer assumptions across different interfaces, bridging methodological design with evaluation practices, and elucidating mechanisms that preserve instructional requirements amid environmental variations. Furthermore, it proposes a research agenda centered on recursive self-improvement, establishing a foundation for compositional task acquisition and faithful physical transfer.

Abstract

General-purpose robots must infer what a new task requires and translate that understanding into appropriate physical action. In-context learning (ICL) for robots supports this process by using demonstrations and interaction to direct existing competence with neural parameters held fixed during deployment. We organize this literature review around the interfaces connecting contextual evidence to execution, distinguishing four families: context-conditioned policies, geometric demonstration transfer, world-model-based control, and skill- and agent-based execution. Comparing these interfaces clarifies their transfer assumptions and the roles of training, correspondence, and memory in making context useful. Across manipulation and navigation, we examine how these mechanisms preserve taught requirements as objects, environments, and execution conditions change. This analysis links method design to evaluation practices that distinguish responsiveness to teaching, physical transfer, and benefits from retained experience. The resulting agenda connects compositional task acquisition and faithful transfer with physical recursive self-improvement, in which experience improves the ability to learn subsequent tasks.

Problem

• In-Context Learning
• General-purpose Robots
• Task Transfer
• Robot Manipulation
• Physical Execution

Innovation

• In-Context Learning
• Robots
• Context-Conditioned Policies
• World-Model-Based Control
• Recursive Self-Improvement