TRaM-VSR: Importance-Aware Token Routing and Merging for One-Step Diffusion Video Super-Resolution

2026-07-24

AI Summary

This work addresses the high computational cost, detail loss, and temporal flickering in video super-resolution caused by dense spatiotemporal tokens in diffusion models. To this end, the authors propose TRaM-VSR, a framework that evaluates token importance by integrating motion-sensitive temporal cues with semantic text similarity and employs an offline planner to guide block-wise routing for multi-granularity token processing. By incorporating context-aware video priors, dynamic object and structural boundary detection, and a dual-stream architecture balancing high and low fidelity, TRaM-VSR achieves state-of-the-art reconstruction quality and robust temporal consistency while significantly improving inference efficiency.

Abstract

Video super-resolution (VSR) using large-scale Diffusion Transformer (DiT) priors achieves exceptional perceptual quality but is often impractical due to the quadratic computational cost of processing dense spatio-temporal token sequences. Existing efficiency-oriented methods risk irreversible detail loss and temporal flickering, a vulnerability especially pronounced in one-step diffusion models. To address this, we propose TRaM-VSR, a Token Routing and Merging framework for adaptive token allocation, leveraging both context-aware video priors and network-level priors. First, token importance is estimated by fusing motion-sensitive temporal cues with semantic text similarity, isolating dynamic objects and structural boundaries. Next, this importance is further calibrated and adjusted by an offline planner to guide routing across optimally grouped network blocks. Technically, within each routed group, structurally critical tokens are processed in a high-fidelity local stream, while less informative tokens are aggregated into a compact global stream, both modulated by network depth and aligned with the multigranular nature of diffusion models. Extensive experiments show that TRaM-VSR accelerates inference significantly while preserving state-of-the-art reconstruction quality and robust temporal consistency. The code is available at https://github.com/Ree1s/TRaM-VSR.

Problem

• video super-resolution
• diffusion models
• computational efficiency
• temporal flickering
• token sequence

Innovation

• Token Routing
• Importance-Aware Merging
• One-Step Diffusion
• Video Super-Resolution
• Efficient DiT