mmHRI: Towards Privacy-Preserving Human-Robot Interaction with Millimeter-Wave Radar

2026-09-28

AI Summary

This study addresses privacy leakage concerns inherent in traditional robot interaction systems that rely on RGB cameras by proposing a millimeter-wave radar-based, privacy-preserving human-robot interaction framework. Methodologically, it introduces the first multimodal radar-guided architecture, incorporating a dual-stream network and a Memory State Space Model (MSSM) to mitigate radar data sparsity, while integrating Vision-Language-Action (VLA) policies for end-to-end task control. Experimental evaluations demonstrate that the proposed system achieves an action recognition accuracy of 85.09% under occluded conditions and successfully executes closed-loop object delivery and retrieval tasks. These findings establish a novel paradigm for robust robotic interaction in privacy-sensitive environments.

Abstract

Assistive robots increasingly operate in many human-centered environments and perform various human-robot interaction (HRI) tasks, such as object delivery. However, most existing HRI systems rely on RGB cameras that continuously observe humans to respond to non-verbal commands, such as hand gestures. This raises privacy concerns in privacy- critical environments, such as hospital wards or restaurants, where direct camera observation of humans is restricted. To develop privacy-preserving HRI, we leverage millimeter-wave (mmWave) radar, which can sense human motion through privacy barriers without identifiable imagery. We propose mmHRI, the first multi-modal robot manipulation framework that achieves mmWave radar-guided privacy-preserving HRI. mmHRI introduces two key designs to mitigate the sparsity and temporal inconsistency of radar data in cluttered robot manipulation environments. First, we propose a dual-stream architecture that jointly learns from unfiltered raw radar tensors and radar point clouds to estimate both human actions and 3D poses. To mitigate signal inconsistency, mmHRI further incorporates a memory-based state-space model (MSSM) that retains historical radar features to reduce abrupt changes in pose/action. These estimated human states are then converted into structured textual robot instructions, which control a vision-language-action (VLA) policy for closed-loop robot manipulation and human-aware reactions. Our evaluation covers human action recognition and closed-loop delivery and retrieval. In the privacy-preserving curtain setting, mmHRI achieves 85.09% action-recognition accuracy, outperforming existing radar-based alternatives. Robot trials further demonstrate successful delivery and retrieval under visual occlusion, with stable task performance across unseen subjects, clutter configurations, and environments.

Problem

• Human-Robot Interaction
• Privacy-Preserving
• Millimeter-Wave Radar
• Action Recognition
• Robot Manipulation

Innovation

• millimeter-wave radar
• privacy-preserving HRI
• dual-stream architecture
• memory-based state-space model
• vision-language-action policy