AV-GRPO: Modality-Anchored Decoupling Diffusion Reinforcement Learning for Joint Audio-Video Generation

2026-09-24

AI Summary

This study addresses the challenges of low fidelity, weak text alignment, and poor cross-modal synchronization in joint audio-video generation, alongside signal entanglement and high computational costs in reinforcement learning post-training. To this end, it proposes the AV-GRPO framework and the 5DAV dataset. The core innovation lies in decoupling multimodal preference learning into unimodal subproblems via modality anchoring, enabling precise reward attribution. Furthermore, this work pioneers an online diffusion RL architecture that substantially reduces computational overhead through a trajectory-locking frozen-tower strategy and an adaptive objective function. By integrating diffusion models, GRPO, and LoRA fine-tuning, the proposed method significantly outperforms the LTX-2.3 baseline on the JavisBench and VABench benchmarks across generation quality, semantic alignment, and cross-modal synchronization.

Abstract

Recent years have witnessed major progress in joint audio-video generation. Existing models still suffer from limited per-modality fidelity, insufficient text-modality alignment and weak cross-modal synchronization. While reinforcement-learning post-training offers a promising remedy, directly adapting it to joint audio-video generation is challenging. Heterogeneous multimodal rewards entangle learning signals and complicate credit assignment. Joint optimization of two modality towers is computationally expensive given their divergent dynamics. Moreover, synchronization evaluation difficulty depends on paired samples, preventing fair reward comparisons. We propose AV-GRPO, a modality-anchored online diffusion RL framework, and 5DAV, a decoupled, difficulty-controllable training dataset. AV-GRPO includes three key modules: (1) modality-anchored rollouts to disentangle learning signals and stabilize difficulty; (2) trajectory-locked frozen-tower optimization to reduce cost and reassign credit; (3) adaptive objectives and perturbation strengths tailored to modality-specific dynamics. This converts coupled multimodal preference learning into unimodal subproblems for precise reward attribution and better synchronization. Our 5DAV dataset decouples samples across five dimensions for systematic training. Experiments on JavisBench and VABench demonstrate AV-GRPO outperforms LTX-2.3 in generation quality, semantic alignment and cross-modal synchronization under LoRA and full fine-tuning. Ablations confirm our designs. Code and data: https://github.com/zhiyuxu03/AV-GRPO

Problem

• Joint Audio-Video Generation
• Reinforcement Learning Post-training
• Cross-modal Synchronization
• Multimodal Credit Assignment
• Modality Fidelity

Innovation

• Modality-Anchored Rollouts
• Decoupled Diffusion Reinforcement Learning
• Trajectory-Locked Optimization
• Cross-Modal Synchronization
• Audio-Video Generation