URNet: A Unified Reparameterized Network for Efficient RGB-D Semantic Segmentation

2026-08-06

AI Summary

This work addresses the limitations of existing RGB-D semantic segmentation methods, which typically employ dual-encoder architectures that suffer from inadequate depth representation, restricted cross-modal interaction, and computational redundancy. To overcome these issues, we propose URNet, a unified framework that processes both RGB and depth inputs through a single encoder to enable efficient multi-scale feature fusion. The core innovations include integrating reparameterized blocks (RepBlocks) with Linear Gated Attention (LGA) modules, allowing simultaneous feature extraction and cross-modal interaction within a unified architecture, as well as designing a lightweight, general-purpose Pyramid Merging Decoder (PMD). Extensive experiments demonstrate that URNet achieves state-of-the-art performance across multiple benchmarks while significantly improving inference efficiency.

Abstract

Previous RGB-D semantic segmentation methods commonly employ dual encoders to separately process RGB and depth inputs, followed by dedicated modules for cross-modal feature fusion. However, such designs often inadequately capture depth representations and consequently limit effective cross-modal interaction, while the additional encoder branch introduces redundant computation that hinders lightweight execution. To tackle these challenges, we propose URNet, a Unified Reparameterized RGB-D Network that performs simultaneous multi-modal feature extraction and cross-modal fusion within a single encoder. Specifically, we adopt a reparameterization strategy to compact the network architecture and facilitate fast inference. Within each Reparameterized Block (RepBlock), a Linear Gated Attention (LGA) module is introduced to fully exploit complementary RGB and depth cues across different feature scales. Furthermore, considering that decoder design has been relatively underexplored in existing RGB-D segmentation models, we develop a concise yet effective universal decoder, termed the Pyramid Merging Decoder (PMD). Extensive experiments on multiple RGB-D segmentation benchmarks demonstrate that URNet achieves state-of-the-art performance while maintaining high efficiency. Code will be available at https://github.com/Wild-Stephen/URNet.

Problem

• RGB-D semantic segmentation
• cross-modal fusion
• depth representation
• model efficiency
• dual encoders

Innovation

• Reparameterization
• Unified Encoder
• Linear Gated Attention
• Pyramid Merging Decoder
• Efficient RGB-D Segmentation