SEO: Stochastic Experience Optimization for Large Language Models

2025-01-08

AI Summary

This work addresses the limited task-specific generalization and heavy reliance on parameter fine-tuning exhibited by large language models (LLMs). To this end, we propose a gradient-free natural language experience optimization method. Our approach introduces an iterative, model-specific self-optimization framework under frozen-parameter conditions; a novel stochastic validation mechanism to ensure the effectiveness and robustness of experience updates; and a model-agnostic experience evaluation and selection strategy enabling cross-distribution data transfer. Experiments across three representative tasks and three mainstream LLMs demonstrate consistent performance gains. Crucially, the optimized experiences generalize effectively to out-of-distribution data and significantly enhance zero-shot transfer capability to semantically similar tasks—without any parameter updates.

Technology Category

• Natural Language Processing: Learning & Optimization for NLP
• Search and Optimization: Learning to Search
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large Language Models (LLMs) can benefit from useful experiences to improve their performance on specific tasks. However, finding helpful experiences for different LLMs is not obvious, since it is unclear what experiences suit specific LLMs. Previous studies intended to automatically find useful experiences using LLMs, while it is difficult to ensure the effectiveness of the obtained experience. In this paper, we propose Stochastic Experience Optimization (SEO), an iterative approach that finds optimized model-specific experience without modifying model parameters through experience update in natural language. In SEO, we propose a stochastic validation method to ensure the update direction of experience, avoiding unavailing updates. Experimental results on three tasks for three LLMs demonstrate that experiences optimized by SEO can achieve consistently improved performance. Further analysis indicates that SEO-optimized experience can generalize to out-of-distribution data, boosting the performance of LLMs on similar tasks.

Problem

• Large Language Models
• Performance Enhancement
• Transfer Learning

Innovation

• Randomized Empirical Optimization (SEO)
• Natural Language Learning
• Generalization Improvement