CLIFT: Turning Gemini Robotics On-Device into Humanoid Specialists via Non-Invasive Closed-Loop Iterative Fine-Tuning

2026-07-31

AI Summary

This work addresses the limitations of closed-weight robotic foundation models, which typically expose only supervised fine-tuning APIs and thus preclude reinforcement learning and other closed-loop policy optimization approaches essential for agile, contact-rich humanoid manipulation tasks. To overcome this constraint without access to model weights or internal training details, the authors propose CLIFT—a non-invasive method that leverages deployment-time reward feedback to generate API-compatible supervised data, enabling iterative closed-loop fine-tuning. Implemented on the Gemini Robotics On-Device platform, CLIFT achieves, for the first time, closed-loop policy improvement on a real humanoid robot without requiring model access, thereby transcending the imitation-learning-only paradigm imposed by hosted APIs. After just two iterations, the refined policy attains near-perfect success rates on challenging contact-intensive tasks, substantially outperforming open-source vision-language-action (VLA) models trained with equivalent data.

Abstract

While robot foundation models are growing increasingly capable, the strongest models are typically trained on proprietary data and remain closed-source, limiting downstream users' ability to adapt them to new tasks, embodiments, and deployment settings. Following the LLM community, an emerging access paradigm for closed-weight robot foundation models is the managed supervised fine-tuning (SFT) API, where users submit training data and receive a tuned policy without access to model weights, gradients, or training internals. While such APIs let downstream users leverage powerful proprietary foundation models, they restrict policy improvement to pure imitation, ruling out reinforcement learning and other closed-loop methods that rely on internal training signals. This limitation is particularly acute for agile, contact-rich humanoid manipulation, where the gap between policy outputs and deployed behavior is large due to novel states, action tracking dynamics, latency, and controller-specific failure modes. We study how effective this managed-API regime is for humanoid adaptation, and how closed-loop improvement can be realized within it to push policies toward task mastery. We conduct one of the first empirical studies of managed-API adaptation on a real humanoid, instantiated on Gemini Robotics On-Device (GROD). We find that direct SFT through the API substantially outperforms a leading open-weight VLA trained on the same demonstrations, yet still falls short of deployment-level mastery on agile, contact-rich tasks. To close this gap, we introduce CLIFT: Closed-Loop Iterative Fine-Tuning, which turns deployment-time reward feedback into API-compatible supervised data and enables closed-loop policy improvement without accessing weights, gradients, likelihoods, or losses-pushing GROD to near-perfect success after two flywheel cycles, all without "opening the model box."

Problem

• closed-loop fine-tuning
• humanoid manipulation
• managed API
• robot foundation models
• on-device adaptation

Innovation

• Closed-loop fine-tuning
• Managed API
• Humanoid robotics
• Non-invasive adaptation
• Policy iteration