A Differentiable Wave Optics Model for End-to-End Computational Imaging System Optimization

2024-12-13

arXiv.org

AI Summary

Existing end-to-end computational imaging optimization methods are computationally prohibitive for jointly modeling diffraction and aberrations in compound optical systems, often neglecting wave-optical effects or off-axis aberrations—thereby compromising design robustness. This work introduces a differentiable wave-optical simulator that, for the first time, enables efficient and accurate joint modeling of diffraction and aberrations within an end-to-end framework, facilitating co-optimization of optical hardware and downstream reconstruction or classification algorithms. Our method leverages differentiable wavefront propagation, physics-driven light-field simulation, and gradient-based parameter updates via backpropagation. Experiments demonstrate that designs optimized solely under geometric optics degrade by 12–28% under realistic wave-optical conditions; in contrast, our co-designed systems achieve significantly enhanced robustness and generalization across both reconstruction and classification tasks.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Hardware-aware ML

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

End-to-end optimization, which simultaneously optimizes optics and algorithms, has emerged as a powerful data-driven method for computational imaging system design. This method achieves joint optimization through backpropagation by incorporating differentiable optics simulators to generate measurements and algorithms to extract information from measurements. However, due to high computational costs, it is challenging to model both aberration and diffraction in light transport for end-to-end optimization of compound optics. Therefore, most existing methods compromise physical accuracy by neglecting wave optics effects or off-axis aberrations, which raises concerns about the robustness of the resulting designs. In this paper, we propose a differentiable optics simulator that efficiently models both aberration and diffraction for compound optics. Using the simulator, we conduct end-to-end optimization on scene reconstruction and classification. Experimental results demonstrate that both lenses and algorithms adopt different configurations depending on whether wave optics is modeled. We also show that systems optimized without wave optics suffer from performance degradation when wave optics effects are introduced during testing. These findings underscore the importance of accurate wave optics modeling in optimizing imaging systems for robust, high-performance applications.

Problem

• Modeling aberration and diffraction in compound optics efficiently
• Addressing performance degradation from neglecting wave optics effects
• Enabling robust end-to-end optimization for computational imaging systems

Innovation

• Differentiable wave optics simulator for compound optics
• End-to-end optimization of lenses and algorithms
• Models both aberration and diffraction efficiently