ReACT-Drug: Reaction-Template Guided Reinforcement Learning for de novo Drug Design

2025-12-24

AI Summary

De novo drug design faces challenges including the vastness of chemical space, synthetic intractability of generated molecules, and poor generalization across diverse protein targets. Method: We propose a target-agnostic, end-to-end reinforcement learning framework featuring: (i) a novel reaction-template-guided Proximal Policy Optimization (PPO) algorithm that constructs a dynamic, chemically grounded action space; (ii) integration of ESM-2 protein embeddings with PDB structural knowledge to enable cross-target transfer; and (iii) biologically informed search-space initialization using fragments from known ligands. ChemBERTa-based molecular encoding and the MOSES evaluation suite are incorporated for robust representation and assessment. Results: Generated molecules achieve 100% chemical validity and novelty, exhibit high predicted binding affinity and excellent synthetic accessibility, and significantly outperform state-of-the-art baselines under multi-objective optimization—establishing a new paradigm for target-generalizable generative drug design.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

De novo drug design is a crucial component of modern drug development, yet navigating the vast chemical space to find synthetically accessible, high-affinity candidates remains a significant challenge. Reinforcement Learning (RL) enhances this process by enabling multi-objective optimization and exploration of novel chemical space - capabilities that traditional supervised learning methods lack. In this work, we introduce extbf{ReACT-Drug}, a fully integrated, target-agnostic molecular design framework based on Reinforcement Learning. Unlike models requiring target-specific fine-tuning, ReACT-Drug utilizes a generalist approach by leveraging ESM-2 protein embeddings to identify similar proteins for a given target from a knowledge base such as Protein Data Base (PDB). Thereafter, the known drug ligands corresponding to such proteins are decomposed to initialize a fragment-based search space, biasing the agent towards biologically relevant subspaces. For each such fragment, the pipeline employs a Proximal Policy Optimization (PPO) agent guiding a ChemBERTa-encoded molecule through a dynamic action space of chemically valid, reaction-template-based transformations. This results in the generation of extit{de novo} drug candidates with competitive binding affinities and high synthetic accessibility, while ensuring 100% chemical validity and novelty as per MOSES benchmarking. This architecture highlights the potential of integrating structural biology, deep representation learning, and chemical synthesis rules to automate and accelerate rational drug design. The dataset and code are available at https://github.com/YadunandanRaman/ReACT-Drug/.

Problem

• Designs novel drugs using reinforcement learning and reaction templates.
• Generates synthetically accessible, high-affinity drug candidates automatically.
• Integrates protein embeddings and chemical rules for target-agnostic optimization.

Innovation

• Reinforcement Learning with PPO for molecular optimization
• Reaction-template guided transformations ensure chemical validity
• ESM-2 embeddings identify similar proteins for fragment initialization