Detection and Recognition: A Pairwise Interaction Framework for Mobile Service Robots

2026-02-25

AI Summary

This work addresses the challenge of enabling mobile service robots to efficiently interpret local human-human interactions for socially aware navigation in human-populated environments, where sensing quality and computational resources are often limited. To this end, the authors propose a two-stage lightweight framework that first detects potential interacting pairs using geometric and motion cues, then classifies coarse-grained interaction behaviors via a relational network. By treating pairwise human interactions as the minimal effective unit of social awareness, the approach avoids overly complex modeling while balancing practicality and computational efficiency. Experiments demonstrate that the model achieves competitive accuracy on the JRDB dataset with significantly lower computational overhead and fewer parameters, and further exhibits strong generalization across the Collective Activity Dataset and a newly collected lawn-mower scenario dataset.

Technology Category

• Intelligent Robots: Human-Robot Interaction
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Autonomous mobile service robots, like lawnmowers or cleaning robots, operating in human-populated environments need to reason about local human-human interactions to support safe and socially aware navigation while fulfilling their tasks. For such robots, interaction understanding is not primarily a fine-grained recognition problem, but a perception problem under limited sensing quality and computational resources. Many existing approaches focus on holistic group activity recognition, which often requires complex and large models which may not be necessary for mobile service robots. Others use pairwise interaction methods which commonly rely on skeletal representations but their use in outdoor environments remains challenging. In this work, we argue that pairwise human interaction constitute a minimal yet sufficient perceptual unit for robot-centric social understanding. We study the problem of identifying interacting person pairs and classifying coarse-grained interaction behaviors sufficient for downstream group-level reasoning and service robot decision-making. To this end, we adopt a two-stage framework in which candidate interacting pairs are first identified based on lightweight geometric and motion cues, and interaction types are subsequently classified using a relation network. We evaluate the proposed approach on the JRDB dataset, where it achieves sufficient accuracy with reduced computational cost and model size compared to appearance-based methods. Additional experiments on the Collective Activity Dataset and zero shot test on a lawnmower-collected dataset further illustrate the generality of the proposed framework. These results suggest that pairwise geometric and motion cues provide a practical basis for interaction perception on mobile service robot providing a promising method for integration into mobile robot navigation stacks in future work. Code will be released soon

Problem

• human-human interaction
• mobile service robots
• pairwise interaction
• social navigation
• interaction recognition

Innovation

• pairwise interaction
• mobile service robots
• lightweight perception
• relation network
• social navigation