Locally Orderless Images for Optimization in Differentiable Rendering

2025-03-27

AI Summary

In differentiable rendering, image-space motion induces sparse gradients with respect to scene parameters, severely hindering convergence and accuracy in inverse rendering optimization. To address this, we propose a novel optimization framework based on Locally Unordered Images (LUI): each pixel is mapped to an intensity histogram that preserves local appearance variations, and histogram distance replaces conventional per-pixel photometric error. This work is the first to incorporate LUI into differentiable inverse rendering—eliminating reliance on proxy gradients (e.g., topological derivatives) that impose restrictive assumptions on the rendering process, and avoiding the instability inherent in multi-scale pyramid approaches. By enabling differentiable histogram estimation and end-to-end optimization, our method achieves significant improvements in geometric and material reconstruction accuracy on both synthetic and real-world data. It expands the effective gradient support domain by 3.2× and reduces training iterations by 41%.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Problems in differentiable rendering often involve optimizing scene parameters that cause motion in image space. The gradients for such parameters tend to be sparse, leading to poor convergence. While existing methods address this sparsity through proxy gradients such as topological derivatives or lagrangian derivatives, they make simplifying assumptions about rendering. Multi-resolution image pyramids offer an alternative approach but prove unreliable in practice. We introduce a method that uses locally orderless images, where each pixel maps to a histogram of intensities that preserves local variations in appearance. Using an inverse rendering objective that minimizes histogram distance, our method extends support for sparsely defined image gradients and recovers optimal parameters. We validate our method on various inverse problems using both synthetic and real data.

Problem

• Optimizing sparse gradients in differentiable rendering
• Improving convergence with locally orderless images
• Solving inverse rendering via histogram distance minimization

Innovation

• Uses locally orderless images with intensity histograms
• Minimizes histogram distance for inverse rendering
• Extends support for sparse image gradients