Problem
Research questions and friction points this paper is trying to address.
neural video compression
implicit neural representations
complexity scalability
rate-distortion performance
video codec
Enhanced Neural Video Representation Compression across Extreme Complexity and Quality Scales
🤖 AI Summary
Existing neural video codecs struggle to balance complexity and scalability: lightweight models suffer from limited rate-distortion performance, while high-performance models exhibit rapidly increasing computational complexity with quality, and lack a unified architecture for multi-scenario deployment. This work proposes NVRC++, a unified neural video coding framework based on implicit neural representations (INRs). By integrating a lightweight INR, multi-resolution feature grids, an efficient overfitting-aware optimization tailored for long videos, and an advanced high-dimensional entropy model, NVRC++ supports four complexity levels (7k–360k MACs/pixel) within a single fixed architecture. Each level spans a broad bitrate range while maintaining real-time decoding. Compared to the state-of-the-art NVRC, NVRC++ achieves up to 7.6× faster decoding at comparable rate-distortion performance, marking the first neural video compression system capable of efficient operation across extreme scales of both complexity and quality.
📝 Abstract
Implicit neural representations (INRs) have recently emerged as a promising approach to video compression, delivering competitive rate-distortion performance alongside rapid decoding. However, existing neural video codecs struggle to balance complexity and scalability. Lightweight models often suffer from degraded compression performance when scaled to different bitrate/quality levels, whereas high-performance models exhibit limited scalability, as their model complexity typically increases with quality. This lack of a unified architecture capable of maintaining consistent complexity across a wide range of bitrates severely limits their diverse real-world deployment. To address these challenges, we introduce NVRC++, a novel INR-based video codec that utilizes a lightweight INR with multiple high-resolution feature grids, providing high scalability at any given complexity level. This is paired with an optimization framework that enables efficient overfitting on high-resolution grids for long video sequences, thereby exploiting spatio-temporal redundancies without prohibitive computational or memory overhead. Additionally, an advanced entropy model is designed for efficiently compressing the high-dimensional grid parameters. As a result, NVRC++ provides four complexity levels (from 7kMACs/pixel to 360kMACs/pixel), each spanning wide bitrate and quality ranges while supporting real-time decoding. The experimental results show that NVRC++ offers a much faster decoding speed (up to 7.6x) compared to the SOTA INR-based video codec, NVRC, while delivering comparable performance.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Implicit Neural Representations
Video Compression
Scalable Codec
Feature Grids
Entropy Modeling
🔎 Similar Papers
No similar papers found.
