PolyLayout: Multi-room Manhattan Layout Estimation

2026-08-04

AI Summary

This work addresses the limitations of existing methods in multi-view, multi-room layout estimation—namely, poor generalization, restrictive geometric assumptions, and neglect of cross-room structural sharing—by proposing a joint optimization framework. The approach parameterizes room layouts as Manhattan-aligned 3D polygons and integrates an explicit camera projection model with pretrained visual features. A neural network predicts quality scores that guide iterative wall splitting and merging operations, augmented by an adaptive polygonal topology refinement mechanism. Evaluated on two newly introduced multi-room, multi-view benchmarks, the method significantly outperforms current state-of-the-art approaches, achieving leading performance in both accuracy and robustness.

Abstract

Estimating room layouts from multi-view imagery is a core task for indoor scene understanding. Existing methods are typically limited either by poor generalization to new datasets or restrictive geometric assumptions of the room shape or camera configuration. Most also estimate rooms independently, failing to exploit shared building structure such as dominant directions, ground plane or ceiling height. We propose PolyLayout, a multi-room layout estimation method that parameterizes room layouts as Manhattan 3D polygons and optimizes them jointly across multiple rooms. The optimization objective is predicted by a neural network on top of robust pre-trained visual features and trained end-to-end with supervision only on output room layouts. At the same time, camera projection and polygon updates remain explicit and model-based. This separation between learned scoring and geometry improves generalization to new datasets and camera parameters. During optimization, PolyLayout adaptively refines the polygon topology through iterative wall split and merge operations while jointly utilizing structural cues across rooms. We introduce two new multi-view multi-room layout benchmarks by providing layout annotations to existing datasets, and experiments show that PolyLayout outperforms prior approaches, both in terms of accuracy and robustness. Project page: https://ghanning.github.io/PolyLayout

Problem

• multi-room layout estimation
• Manhattan layout
• indoor scene understanding
• generalization
• structural cues

Innovation

• Manhattan layout
• multi-room estimation
• 3D polygon parameterization
• joint optimization
• geometry-learning separation