Frequency-Integrated Transformer for Arbitrary-Scale Super-Resolution

2025-04-26

AI Summary

Existing implicit neural representation methods for arbitrary-scale super-resolution (ASSR) neglect frequency-domain modeling, limiting detail recovery and spectral fidelity. This paper introduces the first systematic integration of frequency-domain information into implicit neural representations, proposing a frequency-domain lossless embedding mechanism and a spatial-frequency joint self-attention module—comprising Interaction Implicit Self-Attention and Frequency Correlation Self-Attention—to simultaneously enhance detail representation, improve spectral fidelity, and enable global contextual modeling. Leveraging FFT-based real-imaginary mapping, the method preserves frequency-domain integrity. It achieves state-of-the-art performance across multiple benchmark datasets. Ablation studies and visualization analyses—including feature maps, frequency-domain error maps, and local attribution maps—empirically validate the effectiveness of detail enhancement, spectral consistency, and global context modeling.

Technology Category

• Computer Vision: Representation Learning for Vision
• Machine Learning: Representation Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Methods based on implicit neural representation have demonstrated remarkable capabilities in arbitrary-scale super-resolution (ASSR) tasks, but they neglect the potential value of the frequency domain, leading to sub-optimal performance. We proposes a novel network called Frequency-Integrated Transformer (FIT) to incorporate and utilize frequency information to enhance ASSR performance. FIT employs Frequency Incorporation Module (FIM) to introduce frequency information in a lossless manner and Frequency Utilization Self-Attention module (FUSAM) to efficiently leverage frequency information by exploiting spatial-frequency interrelationship and global nature of frequency. FIM enriches detail characterization by incorporating frequency information through a combination of Fast Fourier Transform (FFT) with real-imaginary mapping. In FUSAM, Interaction Implicit Self-Attention (IISA) achieves cross-domain information synergy by interacting spatial and frequency information in subspace, while Frequency Correlation Self-attention (FCSA) captures the global context by computing correlation in frequency. Experimental results demonstrate FIT yields superior performance compared to existing methods across multiple benchmark datasets. Visual feature map proves the superiority of FIM in enriching detail characterization. Frequency error map validates IISA productively improve the frequency fidelity. Local attribution map validates FCSA effectively captures global context.

Problem

• Enhancing arbitrary-scale super-resolution using frequency domain information
• Integrating frequency data losslessly to improve detail characterization
• Leveraging spatial-frequency relationships for global context capture

Innovation

• FIT integrates frequency information for ASSR
• FIM uses FFT for lossless frequency incorporation
• FUSAM leverages spatial-frequency interrelationship globally