OptimusMesh: Compact Autoregressive Mesh Generation from Point Clouds via Sparse Latent Pivots

2026-10-01

AI Summary

This study addresses the tendency of existing point cloud-to-3D mesh generation methods to produce overly dense meshes with high post-processing costs. To overcome this limitation, this work proposes OptimusMesh, a framework that compresses 2,048 input points into 16 sparse latent pivots, substantially shortening the conditioning sequence. Furthermore, it introduces a two-stage autoregressive architecture that jointly predicts vertices and faces, enabling the direct generation of compact triangular meshes. Compared with state-of-the-art approaches, the proposed method reduces the number of generated mesh faces by 25.7%–94.1%, significantly improving generation efficiency while maintaining competitive geometric fidelity and distribution quality.

Abstract

Generating compact and geometrically faithful 3D meshes directly from point clouds remains a fundamental challenge. Point clouds are unordered and sparse, whereas meshes exhibit irregular structure and varying topology. As a result, many existing approaches rely on implicit representations followed by surface extraction or reconstruction. Although effective, these pipelines can produce dense or over-smoothed meshes, often requiring computationally expensive post-processing and simplification. We present OptimusMesh, a framework for direct compact triangle mesh generation from point clouds using sparse latent pivot conditioning. Our key idea is to compress $2{,}048$ oriented input points into only $16$ sparse latent pivots, reducing the geometric conditioning set by $128\times$. These pivots provide a compact structural representation shared across a two-stage autoregressive framework that first generates mesh vertices and then predicts triangular faces conditioned on the generated vertices and the same pivots. Compared with the evaluated recent point-cloud-conditioned autoregressive methods, which use $257$ decoder-conditioning tokens, OptimusMesh uses only $16$, yielding a $16.1\times$ shorter conditioning sequence. Experiments show that OptimusMesh produces the most compact outputs among the compared recent autoregressive methods, using $25.7\%$--$94.1\%$ fewer faces while maintaining competitive geometric fidelity and distributional quality.

Problem

• Point Clouds
• 3D Mesh Generation
• Autoregressive Generation
• Compact Mesh
• Geometric Fidelity

Innovation

• Autoregressive Mesh Generation
• Sparse Latent Pivots
• Point Clouds
• Compact Triangle Mesh
• Two-stage Framework