ORACLE: A Multi-Objective Reinforcement Learning-Based Analog Circuit Design Optimizer with Large Language Models-Guided Exploration

2026-08-05

AI Summary

This work addresses the challenges of multi-objective Pareto trade-offs and the need for retraining upon specification changes in traditional reinforcement learning for analog circuit design. The authors propose a unified multi-objective reinforcement learning framework that employs vector-valued rewards and a preference-conditioning mechanism, enabling a single model to flexibly adapt to varying objective weight configurations without retraining. To enhance search efficiency, they innovatively integrate a large language model to guide action space pruning and introduce two preference-guided strategies—normalized weighting and cosine alignment. Experimental results demonstrate that the method satisfies 99.9% of specification requirements across 2,000 test cases without retraining, achieving 20.4–104.4× speedup over the state-of-the-art and improving solution quality by 5.1–318.6×.

Abstract

Analog circuit design automation using reinforcement learning (RL) has emerged as a promising approach for reducing manual effort. However, many existing RL-based methods focus on single-objective optimization. Even methods designed for multi-objective (MO) problems often reduce multiple design specifications to a single scalar reward. This simplification limits the ability to capture the true Pareto trade-off among competing objectives and often leads to suboptimal designs. Moreover, requiring the model to be retrained from scratch whenever the desired MO specifications change remains a key limitation. To address these challenges, we present ORACLE, an open-source RL-based framework for MO analog circuit design optimization that replaces scalar reward optimization with vector-valued learning and preference-aware conditioning. ORACLE represents a true MO analog circuit design optimizer that uses a preference vector to specify the relative weights of multiple objectives, enabling a single trained model to generate designs across diverse trade-off settings without retraining. We further propose two preference-guidance strategies, namely normalized-weight guidance and cosine-aligned guidance, to improve convergence. In addition, we incorporate a large language model (LLM)-guided action selection mechanism to filter actions that are likely to lead to suboptimal designs or increased runtime. Our results show that, on multiple circuit topologies with 2,000 test cases, ORACLE reduces runtime by 20.4x - 104.4x compared to state-of-the-art approaches. It also meets 99.9% of the 2,000 target specifications, and achieves 5.1x - 318.6x better figure of merit in the resulting output specs.

Problem

• multi-objective optimization
• analog circuit design
• reinforcement learning
• Pareto trade-off
• design automation

Innovation

• multi-objective reinforcement learning
• preference-aware conditioning
• large language model-guided exploration
• vector-valued reward
• analog circuit design automation