SIDME: Self-supervised Image Demoir'eing via Masked Encoder-Decoder Reconstruction

2025-04-16

AI Summary

This paper addresses moiré pattern degradation induced by camera sampling, proposing a channel-aware self-supervised image demoiréing method. Unlike conventional holistic image modeling approaches, our method explicitly leverages the physical prior of higher sampling rate in the green channel to design a masked encoder-decoder architecture, enabling unsupervised training via random masking and reconstruction. Key contributions include: (1) the first channel-specific modeling framework grounded in sampling-frequency priors; (2) a novel channel-weighted self-supervised loss function that accounts for inter-channel sensitivity to moiré artifacts; and (3) a realistic, self-supervised moiré generation mechanism aligned with actual imaging physics. Evaluated on real-world datasets, our method substantially outperforms existing supervised and self-supervised approaches, demonstrating superior generalization capability and robustness under diverse capture conditions.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Moir'e patterns, resulting from aliasing between object light signals and camera sampling frequencies, often degrade image quality during capture. Traditional demoir'eing methods have generally treated images as a whole for processing and training, neglecting the unique signal characteristics of different color channels. Moreover, the randomness and variability of moir'e pattern generation pose challenges to the robustness of existing methods when applied to real-world data. To address these issues, this paper presents SIDME (Self-supervised Image Demoir'eing via Masked Encoder-Decoder Reconstruction), a novel model designed to generate high-quality visual images by effectively processing moir'e patterns. SIDME combines a masked encoder-decoder architecture with self-supervised learning, allowing the model to reconstruct images using the inherent properties of camera sampling frequencies. A key innovation is the random masked image reconstructor, which utilizes an encoder-decoder structure to handle the reconstruction task. Furthermore, since the green channel in camera sampling has a higher sampling frequency compared to red and blue channels, a specialized self-supervised loss function is designed to improve the training efficiency and effectiveness. To ensure the generalization ability of the model, a self-supervised moir'e image generation method has been developed to produce a dataset that closely mimics real-world conditions. Extensive experiments demonstrate that SIDME outperforms existing methods in processing real moir'e pattern data, showing its superior generalization performance and robustness.

Problem

• Removes moiré patterns from images caused by aliasing
• Handles different color channels' signal characteristics uniquely
• Improves robustness against real-world moiré pattern variability

Innovation

• Masked encoder-decoder architecture for reconstruction
• Self-supervised learning with camera sampling properties
• Specialized loss function for green channel efficiency