KPI: A Promptable Kernel for Physical Interaction on Humanoids

2026-09-28

AI Summary

This study addresses the rigid interaction control caused by insufficient trajectory information in general-purpose whole-body tracking. We propose a novel force-control paradigm based on promptable kernels that decouples trajectory generation from control execution. Specifically, a trajectory source transmits force-control contracts to dynamically modulate stiffness and damping. Furthermore, a vision-language agent is introduced to automatically generate interaction contracts, which, combined with a real-time adaptive algorithm based on tracking error and torque estimation, enables compliant interaction without predefined controllers. Experimental results demonstrate that the system successfully executes tasks such as winch operation, door opening, and object transportation. Notably, a humanoid robot rotates a crank to fully hoist another robot, thereby validating the effectiveness of the proposed approach.

Abstract

Humanoids now walk, balance and reach with remarkable generality: one whole-body tracking policy follows references from a human, or from an end-to-end policy. That generality travels in the trajectory, and a trajectory alone carries limited information about the interaction it should produce: at contact, the executing controller determines how the robot behaves. Single-task policies usually reach hard interactions by optimising trajectory and controller together in simulation; general stacks usually assume a preset or hand-chosen controller. We present KPI, a promptable kernel for physical interaction between the trajectory source and an unmodified whole-body tracker. Instead of a controller fixed before the task, the trajectory source sends a contract: per direction, track, comply, or hold a force range. From tracking error and a wrench estimate, the kernel adapts the arms' stiffness, damping, reference and feedforward toward it at contact rate. We demonstrate KPI through an agentic framework: from one instruction, a vision-language agent writes both the reference trajectory and the contract, with no task-specific code. We demonstrate instruction-driven winch operation, door opening, and box transport, alongside scripted surface-interaction experiments. In the winch demonstration, the humanoid is able to turn a crank to hoist a second robot fully off the ground.

Problem

• humanoid robots
• physical interaction
• whole-body control
• promptable kernel
• generalization

Innovation

• promptable kernel
• physical interaction
• humanoid robots
• vision-language agent
• adaptive impedance control