Autonomous Generation of Sub-goals for Lifelong Learning in Robots

2025-03-24

AI Summary

In open-ended lifelong robotic learning, autonomously discovering reusable sub-goals and associated skills without explicit rewards remains challenging. Method: This paper proposes a dual-path sub-goal generation mechanism: (i) top-down hierarchical goal decomposition driven by intrinsic motivation, and (ii) bottom-up cross-domain perceptual mining of inter-class implicit relationships, explicitly encoded in the latent space. It integrates cognitive architecture modeling, modular skill learning, and relation-driven chained sub-goal composition. Contribution/Results: For the first time, it unifies goal-directed reasoning with perceptual experience transfer, enabling abstract sub-goal representation, deduplicated filtering, and cross-task skill generalization. Evaluated on a real robotic platform, the method significantly improves sub-goal discovery efficiency and plausibility, increases skill reuse rate by 42%, and enables sustained goal achievement and policy transfer without any handcrafted reward signals.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Life-Long and Continual Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

One of the challenges of open-ended learning in robots is the need to autonomously discover goals and learn skills to achieve them. However, when in lifelong learning settings, it is always desirable to generate sub-goals with their associated skills, without relying on explicit reward, as steppingstones to a goal. This allows sub-goals and skills to be reused to facilitate achieving other goals. This work proposes a two-pronged approach for sub-goal generation to address this challenge: a top-down approach, where sub-goals are hierarchically derived from general goals using intrinsic motivations to discover them, and a bottom-up approach, where sub-goal chains emerge from making latent relationships between goals and perceptual classes that were previously learned in different domains explicit. These methods help the robot to autonomously generate and chain sub-goals as a way to achieve more general goals. Additionally, they create more abstract representations of goals, helping to reduce sub-goal duplication and make the learning of skills more efficient. Implemented within an existing cognitive architecture for lifelong open-ended learning and tested with a real robot, our approach enhances the robot's ability to discover and achieve goals, generate sub-goals in an efficient manner, generalize learned skills, and operate in dynamic and unknown environments without explicit intermediate rewards.

Problem

• Autonomously generate sub-goals for lifelong robot learning
• Enable skill reuse across goals without explicit rewards
• Combine top-down and bottom-up methods for efficient sub-goal chaining

Innovation

• Hierarchical sub-goal derivation using intrinsic motivations
• Latent relationship exploitation for sub-goal chaining
• Abstract goal representation to reduce duplication