NExT-Mol: 3D Diffusion Meets 1D Language Modeling for 3D Molecule Generation

2025-02-18

AI Summary

To address the challenges of 3D molecular generation in drug discovery and materials design, this work proposes a two-stage paradigm synergizing 1D language modeling and 3D equivariant diffusion: first, a billion-parameter pretrained SELFIES language model generates 100% chemically valid molecular sequences; second, an enhanced equivariant 3D diffusion model predicts high-accuracy atomic coordinates. Our key innovation lies in enabling 1D→3D cross-modal transfer learning via knowledge distillation and joint scaling to bridge the modality gap. On GEOM-DRUGS, our method achieves a 26% improvement in 3D Fréchet ChemNet Distance (FCD); on QM9-2014 conditional generation, it attains a 13% average improvement over prior methods; and for conformational prediction, it sets a new state-of-the-art. The framework simultaneously ensures molecular validity, distributional fidelity, and geometric accuracy.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

3D molecule generation is crucial for drug discovery and material design. While prior efforts focus on 3D diffusion models for their benefits in modeling continuous 3D conformers, they overlook the advantages of 1D SELFIES-based Language Models (LMs), which can generate 100% valid molecules and leverage the billion-scale 1D molecule datasets. To combine these advantages for 3D molecule generation, we propose a foundation model -- NExT-Mol: 3D Diffusion Meets 1D Language Modeling for 3D Molecule Generation. NExT-Mol uses an extensively pretrained molecule LM for 1D molecule generation, and subsequently predicts the generated molecule's 3D conformers with a 3D diffusion model. We enhance NExT-Mol's performance by scaling up the LM's model size, refining the diffusion neural architecture, and applying 1D to 3D transfer learning. Notably, our 1D molecule LM significantly outperforms baselines in distributional similarity while ensuring validity, and our 3D diffusion model achieves leading performances in conformer prediction. Given these improvements in 1D and 3D modeling, NExT-Mol achieves a 26% relative improvement in 3D FCD for de novo 3D generation on GEOM-DRUGS, and a 13% average relative gain for conditional 3D generation on QM9-2014. Our codes and pretrained checkpoints are available at https://github.com/acharkq/NExT-Mol.

Problem

• Combines 3D diffusion with 1D language modeling
• Enhances 3D molecule generation for drug discovery
• Improves performance in conformer prediction and validity

Innovation

• Combines 1D and 3D molecule modeling
• Enhances with transfer learning techniques
• Improves 3D molecule generation accuracy