RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems

2026-07-31

AI Summary

This work addresses the high cost of manual trial-and-error in recommender system optimization and the instability of autonomous large language model (LLM) search under limited budgets. To this end, the authors propose a two-stage automated evolution framework: a multi-armed bandit router first selects a promising optimization direction based on historical feedback, and then an LLM generates executable hypotheses and code within that direction. A jump-basin mechanism is introduced to escape local stagnation and enable long-horizon exploration. By decoupling routing decisions from code generation, the framework significantly enhances the stability and efficiency of automated tuning. Experiments across multiple datasets and models demonstrate consistent performance gains; in a seven-day online A/B test, the approach increased ad video watch time by 2.084%, revenue by 0.534%, and impressions by 0.559%.

Abstract

Optimizing modern recommender models still depends heavily on engineers manually iterating over architectural, objective, and training-strategy changes. While LLM-based agents can automate this trial-and-error process, allowing the LLM to both select modification directions and generate concrete hypotheses often leads to unstable search under limited experiment budgets. Inspired by the above challenge, we propose RecHarness, a Bandit-Routed Agentic Harness for automated recommender model optimization. RecHarness separates the optimization process into two steps: a bandit router selects the next modification direction according to historical validation feedback, while the LLM generates a concrete optimization hypothesis and executable code edit within the selected direction. To sustain long-horizon exploration, RecHarness uses a jump-basin mechanism to activate a structural-jump arm when local edits stagnate. Across multiple recommendation tasks, datasets, and model backbones, RecHarness achieves more stable performance improvements and uses limited trial budgets more effectively than LLM-reasoning search. During a 7-day online A/B test on a large-scale short-video advertising platform, the selected candidate improves ADVV by 2.084%, Revenue by 0.534%, and Exposure by 0.559%. Code is available at https://github.com/6lyc/RecHarness.

Problem

• recommender systems
• automated optimization
• large language models
• bandit algorithms
• trial-and-error search

Innovation

• Bandit-Routed Optimization
• Self-Evolving Recommender Systems
• LLM-based Code Generation
• Jump-Basin Exploration
• Automated Model Optimization