IE-SRGS: An Internal-External Knowledge Fusion Framework for High-Fidelity 3D Gaussian Splatting Super-Resolution

2025-11-27

AI Summary

To address cross-view texture inconsistency and geometric blurring in reconstructing high-fidelity 3D Gaussian Splatting (3DGS) from low-resolution inputs, this paper proposes an intra- and inter-modal knowledge fusion framework. It leverages pre-trained 2D super-resolution and monocular depth estimation models as external priors while simultaneously exploiting multi-scale internal features of the 3DGS representation. A mask-guided cross-optimization strategy and feature alignment mechanism are introduced to jointly refine geometry and texture. This approach effectively bridges the domain gap between 2D priors and 3D reconstruction, enhancing view consistency and fine-grained detail fidelity. Extensive experiments on both synthetic and real-world datasets demonstrate that our method surpasses state-of-the-art approaches in PSNR, LPIPS, and perceptual visual quality.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Reconstructing high-resolution (HR) 3D Gaussian Splatting (3DGS) models from low-resolution (LR) inputs remains challenging due to the lack of fine-grained textures and geometry. Existing methods typically rely on pre-trained 2D super-resolution (2DSR) models to enhance textures, but suffer from 3D Gaussian ambiguity arising from cross-view inconsistencies and domain gaps inherent in 2DSR models. We propose IE-SRGS, a novel 3DGS SR paradigm that addresses this issue by jointly leveraging the complementary strengths of external 2DSR priors and internal 3DGS features. Specifically, we use 2DSR and depth estimation models to generate HR images and depth maps as external knowledge, and employ multi-scale 3DGS models to produce cross-view consistent, domain-adaptive counterparts as internal knowledge. A mask-guided fusion strategy is introduced to integrate these two sources and synergistically exploit their complementary strengths, effectively guiding the 3D Gaussian optimization toward high-fidelity reconstruction. Extensive experiments on both synthetic and real-world benchmarks show that IE-SRGS consistently outperforms state-of-the-art methods in both quantitative accuracy and visual fidelity.

Problem

• Enhance low-resolution 3D Gaussian Splatting models
• Address 3D Gaussian ambiguity from 2D super-resolution inconsistencies
• Fuse external 2D priors with internal 3D features for fidelity

Innovation

• Fuses external 2DSR priors with internal 3DGS features
• Uses mask-guided fusion to integrate complementary knowledge sources
• Optimizes 3D Gaussians for high-fidelity super-resolution reconstruction