EASE: Embodied Active Event Perception via Self-Supervised Energy Minimization

2025-06-20

AI Summary

Embodied intelligence struggles with active event perception in dynamic, real-world settings lacking predefined action spaces, labeled data, and external rewards. Method: We propose a unified framework grounded in free-energy minimization. It employs prediction error and entropy as intrinsic drives for unsupervised event segmentation, real-time detection, and active tracking. Integrating generative perceptual modeling with action-driven control within a predictive coding architecture, the framework enables self-supervised spatiotemporal representation learning and autonomous decision-making. Contribution/Results: The approach spontaneously exhibits emergent capabilities—including implicit memory, object continuity, and zero-shot environmental adaptation—without explicit supervision. Evaluated on both simulation and physical platforms, it achieves privacy-preserving, scalable event perception without labels or external rewards. It significantly improves robustness and generalization in unscripted, dynamic human-robot collaboration and assistive navigation tasks.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Motion & Tracking

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Active event perception, the ability to dynamically detect, track, and summarize events in real time, is essential for embodied intelligence in tasks such as human-AI collaboration, assistive robotics, and autonomous navigation. However, existing approaches often depend on predefined action spaces, annotated datasets, and extrinsic rewards, limiting their adaptability and scalability in dynamic, real-world scenarios. Inspired by cognitive theories of event perception and predictive coding, we propose EASE, a self-supervised framework that unifies spatiotemporal representation learning and embodied control through free energy minimization. EASE leverages prediction errors and entropy as intrinsic signals to segment events, summarize observations, and actively track salient actors, operating without explicit annotations or external rewards. By coupling a generative perception model with an action-driven control policy, EASE dynamically aligns predictions with observations, enabling emergent behaviors such as implicit memory, target continuity, and adaptability to novel environments. Extensive evaluations in simulation and real-world settings demonstrate EASE's ability to achieve privacy-preserving and scalable event perception, providing a robust foundation for embodied systems in unscripted, dynamic tasks.

Problem

• Dynamic event detection without predefined action spaces
• Self-supervised learning for real-time event perception
• Adaptable event tracking in novel environments

Innovation

• Self-supervised energy minimization framework
• Generative perception with action-driven policy
• Intrinsic signals for event segmentation