Less Is More: Genetic Frame Selection for Efficient Novel View Synthesis

2026-09-28

AI Summary

This study addresses the issue that redundant views increase computational overhead and degrade reconstruction quality in feed-forward novel view synthesis. We propose a lightweight, rendering-free keyframe selector that integrates geometric and image features to construct a multi-criteria scoring system based on coverage, redundancy, and clarity. By employing a genetic algorithm for offline optimal subset search and knowledge distillation to train a compact network, our method breaks away from the conventional selection paradigm that relies on reconstruction feedback. Experiments across six datasets demonstrate that the proposed approach outperforms existing baselines while significantly reducing selection costs. Notably, the curated subsets achieve superior performance compared to full-sequence inputs and exhibit strong cross-paradigm generalization capabilities.

Abstract

Feed-forward novel view synthesis reconstructs a scene from many input images in a single forward pass, yet more views do not necessarily improve performance: redundant or poorly chosen frames increase computational cost and may degrade reconstruction quality. We address the problem of selecting, from an already captured sequence, a fixed-size subset of input views that is most informative for reconstructing specified target viewpoints. We propose a render-free view selector that scores candidate frames based on three complementary criteria: target-view coverage, measured against observed frames that stand in for the targets, redundancy with previously selected views, and image sharpness. A lightweight scoring network then selects the most informative frames without rendering, reconstruction, or per-scene optimization at inference time. To train the selector, we distill an expensive offline search procedure in which a genetic algorithm identifies high-quality subsets by directly optimizing reconstruction performance on training scenes. The selector learns to reproduce these choices from geometric and image-level features alone. Across six datasets and multiple input budgets, our method consistently outperforms both geometric and reconstruction-aware view-selection baselines while incurring significantly lower selection costs than reconstruction-based alternatives. Moreover, carefully selected subsets can outperform feed-forward reconstruction from the full input sequence. The learned selector generalizes across diverse reconstruction paradigms (feed-forward, 3D Gaussian Splatting, and NeRF), to object-targeted reconstruction and to a cross-capture setting in which the target views come from a separate acquisition pass. More broadly, our results indicate that explicitly reasoning about target relevance and inter-view redundancy is a fundamental factor in efficient scene reconstruction.

Problem

• Novel View Synthesis
• View Selection
• Feed-forward Reconstruction
• Frame Selection

Innovation

• Novel View Synthesis
• View Selection
• Genetic Algorithm
• Knowledge Distillation
• Render-free Selector