TreeRef-BFN: Equivariance-Free De Novo Molecule Generation based on 2D Topology and Internal 3D Geometry

2026-09-26

AI Summary

This study addresses the limitations of existing 3D molecular generation methods, which rely on pre-sampled sizes and equivariant architectures, while internal coordinate approaches suffer from error accumulation. We propose TreeRef, a tree-based topology-geometry coupled representation integrated with Bayesian Flow Networks (BFNs) and standard Transformers, enabling joint unconditional and structure-conditioned generation via masking mechanisms. This framework pioneers an equivariance-free molecular representation where molecular sizes emerge naturally through empty nodes, complemented by RingRef ring encoding for enhanced structural expressiveness. A single model supports multi-task generation, significantly improving chemical validity, stability, and diversity. By precisely modeling local geometric distributions, our approach achieves rapid sampling and competitive property-guided generation.

Abstract

De novo 3D molecular generation jointly models molecular size, topology, and geometry. Most methods pre-sample molecular size and generate Cartesian coordinates, limiting variable-size conditional tasks such as fragment completion and scaffold decoration while often relying on equivariant architectures. Internal-coordinate methods avoid rigid-body redundancy but typically require a known molecular graph or autoregressive construction, which may accumulate errors. We propose TreeRef, a tree-based molecular representation that assigns molecular topology and topology-dependent local 3D geometry to a naturally variable-size tree. RingRef nodes encode ring closures while preserving the tree structure, while Null nodes allow molecular size to emerge directly from node occupancy. Based on TreeRef, we develop TreeRef-BFN, a Bayesian Flow Network with a standard Transformer backbone that globally couples these locally defined variables and jointly generates discrete molecular variables and continuous local geometry. A single pretrained TreeRef-BFN supports unconditional generation and variable-size structure-conditioned 3D generation through masking alone, without retraining. Empirical studies demonstrate strong chemical validity, molecular stability, and diversity, accurate local geometric distributions, fast sampling, and competitive property-conditioned generation, establishing TreeRef-BFN as an efficient and flexible framework for 3D molecular generation.

Problem

• De novo 3D molecular generation
• variable-size conditional generation
• internal coordinates
• molecular topology
• error accumulation

Innovation

• De novo molecule generation
• Tree-based representation
• Bayesian Flow Network
• Internal 3D geometry
• Equivariance-free