Problem
Research questions and friction points this paper is trying to address.
geometry-consistent generation
latent space
visual generators
perception models
3D coherence
GAE: Learning a Geometry-Native Latent Space for 3D-Consistent World Generation
๐ค AI Summary
็ ็ฉถๆๅบๅ ไฝๅ็่ชๅจ็ผ็ ๅจ(GAE)่งฃๅณ3Dๅบๆฏไธ่ดๆง็ๆ้ฎ้ข๏ผ้่ฟๆๅปบ็ดงๅ็ๅ ไฝๅ็ๆฝๅจ็ฉบ้ดๆฅๆ้ซ็ๆๅพๅ็่ดจ้ๅ3Dไธ่ดๆงใ
๐ Abstract
We present a compact geometry-native latent space as a shared foundation for perception and generation. Visual generators can produce photorealistic frames without preserving a consistent 3D scene. We argue that this is not only a modeling problem but also a representation problem: generators typically evolve appearance-centric latents, while perception models recover geometry in a semantically rich space that encodes cross-view structure. Rather than adding geometry as another output, we reparameterize a geometry foundation model's features into a compact latent space for generation. We realize this shift with the geometry-native autoencoder (GAE), whose latent is jointly decodable to appearance, depth, cameras, and point maps. With this state, a standard conditional flow supports diverse generation tasks. In controlled comparisons that hold the generator and training protocol fixed, replacing the latent with GAE improves both visual quality and independently measured 3D coherence: FVD falls by $12.7\%$ and $23.1\%$ on RealEstate10K and DL3DV, and camera-trajectory error is halved on RealEstate10K. Together, these results show that the latent space is central to geometry-consistent generation and can serve as a shared interface between perception and generation.
Innovation
Methods, ideas, or system contributions that make the work stand out.
geometry-native latent space
jointly decodable
3D-consistent generation
๐ Similar Papers
No similar papers found.
๐ผ Related Jobs
No related jobs found.
