ADO-LLM: Analog Design Bayesian Optimization with In-Context Learning of Large Language Models

2024-06-26

arXiv.org

AI Summary

Analog circuit design heavily relies on expert knowledge, while conventional Bayesian optimization (BO) suffers from low sample efficiency and insufficient exploration in high-dimensional design spaces, hindering automation. This paper proposes the first synergistic framework integrating large language models (LLMs) with BO: an LLM leverages domain expertise to generate high-quality initial candidate designs; BO iteratively refines them via Gaussian process modeling and simulation feedback; and closed-loop feedback enhances the LLM’s contextual understanding and output diversity. This bidirectional collaboration overcomes key limitations—namely, the lack of prior knowledge in purely data-driven BO and the absence of validation in prompt-engineered LLMs. Evaluated on two representative analog circuit design tasks, our method accelerates convergence by 2.3× and improves Pareto-front coverage by 37%, significantly reducing manual trial-and-error and simulation overhead.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Analogy

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

Analog circuit design requires substantial human expertise and involvement, which is a significant roadblock to design productivity. Bayesian Optimization (BO), a popular machine learning based optimization strategy, has been leveraged to automate analog design given its applicability across various circuit topologies and technologies. Traditional BO methods employ black box Gaussian Process surrogate models and optimized labeled data queries to find optimization solutions by trading off between exploration and exploitation. However, the search for the optimal design solution in BO can be expensive from both a computational and data usage point of view, particularly for high dimensional optimization problems. This paper presents ADO-LLM, the first work integrating large language models (LLMs) with Bayesian Optimization for analog design optimization. ADO-LLM leverages the LLM's ability to infuse domain knowledge to rapidly generate viable design points to remedy BO's inefficiency in finding high value design areas specifically under the limited design space coverage of the BO's probabilistic surrogate model. In the meantime, sampling of design points evaluated in the iterative BO process provides quality demonstrations for the LLM to generate high quality design points while leveraging infused broad design knowledge. Furthermore, the diversity brought by BO's exploration enriches the contextual understanding of the LLM and allows it to more broadly search in the design space and prevent repetitive and redundant suggestions. We evaluate the proposed framework on two different types of analog circuits and demonstrate notable improvements in design efficiency and effectiveness.

Problem

• Automating analog circuit design to enhance productivity
• Reducing computational and data costs in Bayesian Optimization
• Integrating LLMs with BO for efficient design optimization

Innovation

• Integrates LLMs with Bayesian Optimization for analog design
• Uses LLMs to generate viable design points efficiently
• Enhances BO exploration with LLM's contextual understanding