C-MORAL: Controllable Multi-Objective Molecular Optimization with Reinforcement Alignment for LLMs

2026-04-24

AI Summary

This work addresses the challenge of balancing multiple competing constraints in molecular optimization with large language models (LLMs). It proposes the first reinforcement learning–based post-training framework for controllable multi-objective molecular generation, integrating grouped relative optimization, alignment of property scores across heterogeneous objectives, and continuous nonlinear reward aggregation. By aligning LLMs with continuous multi-objective drug design requirements through reinforcement learning post-training, the method significantly enhances optimization stability across competing properties. Evaluated on the C-MuMOInstruct benchmark, it achieves optimization rates of 48.9% on in-distribution (IND) tasks and 39.5% on out-of-distribution (OOD) tasks, while effectively preserving molecular scaffold similarity.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large language models (LLMs) show promise for molecular optimization, but aligning them with selective and competing drug-design constraints remains challenging. We propose C-Moral, a reinforcement learning post-training framework for controllable multi-objective molecular optimization. C-Moral combines group-based relative optimization, property score alignment for heterogeneous objectives, and continuous non-linear reward aggregation to improve stability across competing properties. Experiments on the C-MuMOInstruct benchmark show that C-Moral consistently outperforms state-of-the-art models across both in-domain and out-of-domain settings, achieving the best Success Optimized Rate (SOR) of 48.9% on IND tasks and 39.5% on OOD tasks, while largely preserving scaffold similarity. These results suggest that RL post-training is an effective way to align molecular language models with continuous molecular design objectives. Our code and models are publicly available at https://github.com/Rwigie/C-MORAL.

Problem

• molecular optimization
• multi-objective alignment
• drug design constraints
• large language models
• reinforcement learning

Innovation

• reinforcement learning
• multi-objective optimization
• molecular design
• large language models
• reward aggregation