Asynchronous Multimodal Diffusion Policy Composition via Latency-Aware Guidance Fusion

2026-07-19

AI Summary

This work addresses the challenge of asynchronous perception frequencies and inference latencies across modalities in multimodal diffusion policies for robotic imitation learning, where conventional synchronous fusion methods struggle to balance high-frequency responsiveness with system scalability. The authors propose LAG-Fusion, a novel framework that enables delay-aware asynchronous composition of multimodal diffusion policies: each modality operates independently at its native frequency and provides denoising guidance whenever available. Consistency in fusion is ensured through a reference-frame rebasing rule under a relative action representation, which aligns asynchronous guidance signals. Without requiring custom multi-rate architectures, LAG-Fusion flexibly integrates arbitrary modalities—demonstrating superior performance by fusing low-frequency vision with high-frequency force feedback in tactile-rich tasks, significantly outperforming synchronous fusion and specialized force-based baselines in both response speed and task success under heterogeneous delays.

Abstract

Diffusion policies have shown strong potential for robotic imitation learning, and recent extensions incorporate additional modalities to improve manipulation performance. However, these modalities often differ not only in information content but also in sensing rates and inference latencies. Existing multimodal diffusion policies typically rely on synchronous fusion or manually designed multi-frequency architectures, which either slow down high-frequency feedback or limit extensibility to new modality combinations. We propose LAG-Fusion, a latency-aware guidance fusion framework for asynchronous multimodal diffusion policy composition. LAG-Fusion allows modality-specific policies to operate at their native inference rates and contribute denoising guidance whenever available. To make asynchronous composition consistent, we derive a reference-frame rebasing rule for diffusion variables under relative action representations, enabling delayed guidance to be aligned before fusion. We instantiate LAG-Fusion in contact-rich manipulation by composing a low-frequency vision policy with a high-frequency force policy. Experiments under heterogeneous modality latencies show that LAG-Fusion improves policy responsiveness and task performance over synchronous fusion and specially designed force-aware baselines.

Problem

• asynchronous multimodal
• diffusion policy
• latency-aware fusion
• inference latency
• modality synchronization

Innovation

• asynchronous multimodal fusion
• diffusion policy
• latency-aware guidance
• reference-frame rebasing
• robotic imitation learning