ConTraIRL: Factorized Contrastive Abstractions for Transferable IRL

2026-06-01

AI Summary

This work addresses the challenge of reward transfer in inverse reinforcement learning when faced with unseen combinations of environment dynamics and task objectives. To enable reliable and compositional reward inference, the authors propose a dual-encoder architecture that leverages contrastive learning and temporal alignment to disentangle and separately encode representations of dynamics and task goals in a latent space. This decoupling facilitates few-shot transfer to novel dynamics–objective pairings, significantly improving both sample efficiency and reward recovery performance on continuous control benchmarks.

Abstract

Reward transfer in Inverse Reinforcement Learning (IRL) is unreliable when policies must generalize to unseen combinations of environment dynamics and task goals. We propose Factorized Contrastive Abstractions for Transferable IRL (ConTraIRL), a framework that enables compositional reward transfer by learning decoupled latent representations of these two factors. ConTraIRL uses a dual-encoder architecture that maps observations into separate dynamics and goal latent spaces, trained with a dual contrastive objective. Temporal alignment encourages the dynamics encoder to learn goal-invariant structure, while the goal encoder captures dynamics-invariant features. This factorization supports reward inference under recombined dynamics-goal settings. Experiments on continuous control benchmarks demonstrate effective few-shot transfer to unseen dynamics-goal pairings, improving sample efficiency and reward recovery over transfer IRL baselines.

Problem

• reward transfer
• Inverse Reinforcement Learning
• environment dynamics
• task goals
• compositional generalization

Innovation

• Factorized Contrastive Abstractions
• Transferable IRL
• Decoupled Latent Representations
• Dual-Encoder Architecture
• Compositional Reward Transfer