MINR: Implicit Neural Representations with Masked Image Modelling

2025-07-30

AI Summary

Existing masked autoencoders (e.g., MAE) rely heavily on predefined masking strategies and exhibit limited generalization to out-of-distribution (OOD) data. Method: This paper proposes MINR, the first framework to integrate implicit neural representations (INRs) into masked image modeling. MINR models images as continuous mappings from spatial coordinates to pixel values, enabling geometrically aware structural modeling and enforcing smoothness priors. This continuous functional formulation inherently reduces dependence on specific masking schemes, enhances reconstruction stability, improves OOD robustness, and lowers parameter count. Contribution/Results: Experiments demonstrate that MINR consistently outperforms MAE both in-domain and across diverse OOD scenarios, achieving superior reconstruction fidelity, generalization capability, and transfer performance on downstream tasks. These results validate the effectiveness and universality of continuous function modeling for self-supervised visual representation learning.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Representation Learning for Vision
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Self-supervised learning methods like masked autoencoders (MAE) have shown significant promise in learning robust feature representations, particularly in image reconstruction-based pretraining task. However, their performance is often strongly dependent on the masking strategies used during training and can degrade when applied to out-of-distribution data. To address these limitations, we introduce the masked implicit neural representations (MINR) framework that synergizes implicit neural representations with masked image modeling. MINR learns a continuous function to represent images, enabling more robust and generalizable reconstructions irrespective of masking strategies. Our experiments demonstrate that MINR not only outperforms MAE in in-domain scenarios but also in out-of-distribution settings, while reducing model complexity. The versatility of MINR extends to various self-supervised learning applications, confirming its utility as a robust and efficient alternative to existing frameworks.

Problem

• Improves robustness to varying masking strategies in self-supervised learning
• Enhances generalization for out-of-distribution data in image reconstruction
• Reduces model complexity while maintaining or improving performance

Innovation

• Combines implicit neural representations with masking
• Learns continuous functions for robust reconstructions
• Reduces model complexity and improves generalization