MDNF: Multi-Diffusion-Nets for Neural Fields on Meshes

2024-09-04

arXiv.org

AI Summary

To address challenges in neural field modeling on triangular meshes—including multi-resolution representation, discontinuity handling, exponential-scale variations, and robustness to mesh perturbations—this paper proposes a spatio-temporal and spectral joint multi-resolution representation framework. Methodologically, it introduces the first integration of geometry-aware multi-scale DiffusionNet with a hierarchical Neural Fourier Filter Bank (NFFB), enabling wavelet-like neural field synthesis directly on graphs. Coupled with sine-activated MLPs and multi-scale graph Fourier feature mapping, the framework unifies fine-grained spatial structure and high-frequency signal modeling. Evaluated across diverse neural field tasks—including RGB color, UV coordinate, and vertex normal regression—the method achieves state-of-the-art accuracy and generalization. Notably, it demonstrates exceptional robustness under severe mesh deformations and abrupt scale changes, outperforming existing baselines while maintaining reconstruction fidelity and structural consistency.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Computer Vision: Diffusion Models for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

We propose a novel framework for representing neural fields on triangle meshes that is multi-resolution across both spatial and frequency domains. Inspired by the Neural Fourier Filter Bank (NFFB), our architecture decomposes the spatial and frequency domains by associating finer spatial resolution levels with higher frequency bands, while coarser resolutions are mapped to lower frequencies. To achieve geometry-aware spatial decomposition we leverage multiple DiffusionNet components, each associated with a different spatial resolution level. Subsequently, we apply a Fourier feature mapping to encourage finer resolution levels to be associated with higher frequencies. The final signal is composed in a wavelet-inspired manner using a sine-activated MLP, aggregating higher-frequency signals on top of lower-frequency ones. Our architecture attains high accuracy in learning complex neural fields and is robust to discontinuities, exponential scale variations of the target field, and mesh modification. We demonstrate the effectiveness of our approach through its application to diverse neural fields, such as synthetic RGB functions, UV texture coordinates, and vertex normals, illustrating different challenges. To validate our method, we compare its performance against two alternatives, showcasing the advantages of our multi-resolution architecture.

Problem

• Represent neural fields on meshes with multi-resolution spatial and frequency domains
• Achieve geometry-aware decomposition using DiffusionNet components
• Handle discontinuities and scale variations in complex neural fields

Innovation

• Multi-resolution neural fields on meshes
• DiffusionNet for spatial-frequency decomposition
• Wavelet-inspired signal composition with MLP