Reinforcement Learning: From Algorithms To Foundation Models

2026-07-20

AI Summary

This work addresses the challenge of achieving efficient and robust sequential decision-making in complex dynamic environments by advancing reinforcement learning from both algorithmic and foundational model perspectives. It investigates strategic learning and equilibrium behaviors in multi-agent games while integrating generative and foundation models to enhance environment modeling and incorporate prior knowledge. The study innovatively employs diffusion models for world modeling, represents policies through generative models, and constructs an interactive video world model capable of action intervention. A memory architecture is further introduced to support long-horizon decision-making. Experiments demonstrate that the proposed approach significantly improves sample efficiency, generalization, and long-horizon control stability in both multi-agent competitive settings and high-dimensional video-based tasks.

Abstract

Reinforcement learning (RL) provides a framework for sequential decision making under explicit objectives. In its classical form, RL studies how an agent should act to maximise long-term reward in a dynamic environment. In richer settings, the problem extends beyond a single agent and fixed environment: intelligent behavior may require strategic interaction, adaptation to uncertainty, and reasoning over high-dimensional worlds. This thesis studies RL from two perspectives: algorithms in games and RL in the era of foundation models. The first part focuses on multi-agent RL in games. It examines how incentives, policies, and equilibrium concepts interact in competitive and general-sum environments, spanning two-player zero-sum games, large-scale video games, and multi-player settings with general structure. These works investigate learning in multi-agent systems and the behavior of RL methods in interactive environments. The second part studies RL with generative and foundation models, motivated by the idea that prior knowledge can enrich sequential decision making. Pretrained generative models and learned world models serve as representation tools and structured priors for planning, control, and policy optimization. The thesis develops diffusion-based world models, investigates RL for efficient video generation, explores generative models as policy classes, and studies interactive video world models in which actions shape future observations. It also addresses long-horizon modeling through architectures with memory. Together, these contributions present a unified view of RL as objective-driven adaptation in complex sequential domains. From strategic games to generative world models, the thesis highlights how RL connects decision making, environment modeling, and emerging foundation-model capabilities, offering a broader perspective on the principles underlying intelligent behavior.

Problem

• Reinforcement Learning
• Multi-agent Systems
• Foundation Models
• Generative Models
• Sequential Decision Making

Innovation

• multi-agent reinforcement learning
• foundation models
• generative world models
• diffusion-based planning
• interactive video generation