Enhancing LLMs for Power System Simulations: A Feedback-driven Multi-agent Framework

2024-11-21

arXiv.org

AI Summary

To address critical bottlenecks in power system simulation—namely, domain-knowledge scarcity, weak logical reasoning, and imprecise parameter control in large language models (LLMs), this paper proposes a feedback-driven multi-agent framework. Methodologically, it introduces a novel dynamic environment execution module integrated with real-time error feedback, coupled with an enhanced RAG (Retrieval-Augmented Generation) and chain-of-thought–refinement hybrid reasoning architecture, enabling closed-loop self-correction and low-cost response for simulation tasks. Evaluated on 69 benchmark simulation tasks across Daline and MATPOWER, the framework achieves success rates of 93.13% and 96.85%, respectively, with an average latency of 30 seconds and cost of only $0.014 per query—substantially outperforming baselines including ChatGPT-4o. This work establishes a new paradigm for LLM-powered, high-fidelity, and trustworthy autonomous power system simulation.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Large Multimodal Models (LMMs)
• Multiagent Systems: Multiagent Learning

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

The integration of experimental technologies with large language models (LLMs) is transforming scientific research. It positions AI as a versatile research assistant rather than a mere problem-solving tool. In the field of power systems, however, managing simulations -- one of the essential experimental technologies -- remains a challenge for LLMs due to their limited domain-specific knowledge, restricted reasoning capabilities, and imprecise handling of simulation parameters. To address these limitations, this paper proposes a feedback-driven, multi-agent framework. It incorporates three proposed modules: an enhanced retrieval-augmented generation (RAG) module, an improved reasoning module, and a dynamic environmental acting module with an error-feedback mechanism. Validated on 69 diverse tasks from Daline and MATPOWER, this framework achieves success rates of 93.13% and 96.85%, respectively. It significantly outperforms ChatGPT 4o, o1-preview, and the fine-tuned GPT-4o, which all achieved a success rate lower than 30% on complex tasks. Additionally, the proposed framework also supports rapid, cost-effective task execution, completing each simulation in approximately 30 seconds at an average cost of 0.014 USD for tokens. Overall, this adaptable framework lays a foundation for developing intelligent LLM-based assistants for human researchers, facilitating power system research and beyond.

Problem

• Enhancing LLMs for power system simulation challenges
• Improving domain-specific knowledge and reasoning in LLMs
• Optimizing simulation parameter handling with feedback mechanisms

Innovation

• Feedback-driven multi-agent framework for LLMs
• Enhanced RAG and reasoning modules
• Dynamic environmental acting with error-feedback