An Introduction to Flow Matching and Diffusion Models

2025-06-02

AI Summary

Existing pedagogical approaches to generative AI instruction suffer from a theoretical fragmentation between diffusion models and flow matching, resulting in conceptual gaps when bridging foundational principles to state-of-the-art models (e.g., Sora, DALL·E 3). Method: We propose the first unified teaching framework that systematically integrates both paradigms under a common stochastic differential equation (SDE) and probabilistic flow matching perspective, covering core mechanisms—including score matching, denoising training, and classifier-free guidance—while tightly coupling mathematical rigor (ODE/SDE derivations) with engineering practice (image and video generation tasks). Contribution/Results: The framework establishes a self-consistent, extensible, principle-driven curriculum that bridges systematic theoretical understanding with industrial-scale model comprehension. It has been successfully deployed in MIT’s 2025 Independent Activities Period (IAP) course, demonstrably enhancing learners’ modeling intuition and hands-on proficiency in generative AI.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Diffusion and flow-based models have become the state of the art for generative AI across a wide range of data modalities, including images, videos, shapes, molecules, music, and more! These notes are originally from https://diffusion.csail.mit.edu/, as taught at MIT over the 2025 IAP (winter) term, and are intended to accompany other course content, including lectures and labs. Overall, they function as a self-contained introduction to both flow matching and diffusion models, starting with ordinary and stochastic differential equations, and culminating in flow matching, score matching, classifier-free guidance, and the inner workings of modern, state-of-the-art models for image and video. These notes, and the accompanying course, are ideal for students and practitioners alike who want to develop a principled understanding of the theory and practice of generative AI.

Problem

• Introducing flow matching and diffusion models for generative AI
• Explaining theory and practice of modern generative models
• Providing educational resources for students and practitioners

Innovation

• Flow matching for generative AI
• Score matching techniques
• Classifier-free guidance models