Can Large Language Models Invent Algorithms to Improve Themselves?

2024-10-21

arXiv.org

AI Summary

Can large language models (LLMs) autonomously invent and iteratively optimize their own algorithms without human intervention? Method: We propose the Self-Developing framework, which integrates meta-reasoning, program synthesis, online self-training, and evaluation-driven feedback to enable LLMs to autonomously search for, generate, apply, and iteratively refine model fusion algorithms on mathematical reasoning tasks. Contribution: This work presents the first demonstration of LLMs autonomously discovering fusion strategies that surpass human-designed baselines. The framework supports co-evolution of algorithms and models and exhibits strong cross-domain generalization. On GSM8K, it achieves a +6.0% absolute accuracy gain over the seed model—outperforming state-of-the-art hand-crafted fusion methods by +4.3%. Moreover, it improves out-of-domain model transfer performance by up to +7.4%, confirming its robust adaptability across heterogeneous model families.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search
• Natural Language Processing: (Large) Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Large Language Models (LLMs) have shown remarkable performance improvements and are rapidly gaining adoption in industry. However, the methods for improving LLMs are still designed by humans, which restricts the invention of new model-improving algorithms to human expertise and imagination. To address this, we propose the Self-Developing framework, which enables LLMs to autonomously generate and learn model-improvement algorithms. In this framework, the seed model generates, applies, and learns model-improving algorithms, continuously improving both the seed model and the algorithms themselves. Among model-improving strategies, we focus on model merging algorithms. In mathematical reasoning tasks, Self-Developing discovers novel merging strategies and outperforms human-designed methods. On GSM8k, the discovered algorithms improve the seed model by 6% and surpass human-designed methods by 4.3%. Moreover, they exhibit strong transferability, achieving a 7.4% performance gain on out-of-domain models. These results suggest that LLMs can autonomously develop effective model-improvement techniques beyond human intuition.

Problem

• LLMs autonomously generate improvement algorithms
• Self-Developing framework enhances model performance
• Novel strategies outperform human-designed methods

Innovation

• LLMs autonomously generate improvement algorithms
• Self-Developing framework enhances model performance
• Novel merging strategies outperform human-designed methods