🤖 AI Summary
This work addresses the limited capacity of existing distributional reinforcement learning (DRL) methods to model the full state-action return distribution. We propose Implicit Generative Networks (IGN), the first approach to integrate Generative Adversarial Network (GAN) architecture into the Implicit Quantile Network (IQN) framework: a generator implicitly models the quantile function, while a discriminator assists in calibrating distributional fidelity; combined with quantile regression, IGN enables end-to-end, high-fidelity approximation of the entire return distribution. Crucially, IGN abandons conventional explicit quantile sampling, thereby significantly enhancing distributional representation capability and risk-sensitive policy optimization. Evaluated across all 57 Atari 2600 games, IGN consistently outperforms prior DRL methods in distributional robustness, mean score, and risk controllability—achieving state-of-the-art performance in both risk-sensitive training and evaluation.
📝 Abstract
In this work, we build recent advances in distributional reinforcement learning to give a state-of-art distributional variant of the model based on the IQN. We achieve this by using the GAN model’s generator and discriminator function with the quantile regression to approximate the full quantile value for the state-action return distribution. We demonstrate improved performance on our baseline dataset - 57 Atari 2600 games in the ALE. Also, we use our algorithm to show the state-of-art training performance of risk-sensitive policies in Atari games with the policy optimization and evaluation.
