OMS: On-the-fly, Multi-Objective, Self-Reflective Ad Keyword Generation via LLM Agent

2025-07-03

AI Summary

To address the limitations of keyword selection in sponsored search advertising—namely, overreliance on historical data, absence of online multi-objective optimization, and insufficient quality control—this paper proposes a training-free LLM-based agent framework. The method employs an on-the-fly generation mechanism that dynamically synthesizes keywords by integrating real-time performance signals (e.g., impressions, CTR, CVR, CTA efficacy), multi-objective reward modeling, and self-reflective quality assessment. Its core contributions include: (1) eliminating dependence on large-scale query-keyword pairs; (2) enabling online monitoring and autonomous reasoning; and (3) introducing a self-evaluation-driven closed-loop optimization that significantly improves keyword relevance and advertising performance. Extensive evaluations on benchmark datasets and live advertising campaigns demonstrate consistent superiority over state-of-the-art methods. Ablation studies and human evaluations further validate the effectiveness and quality of each component.

Technology Category

• Search and Optimization: Learning to Search
• Multiagent Systems: Adversarial Agents
• Machine Learning: Online Learning & Bandits

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Keyword decision in Sponsored Search Advertising is critical to the success of ad campaigns. While LLM-based methods offer automated keyword generation, they face three major limitations: reliance on large-scale query-keyword pair data, lack of online multi-objective performance monitoring and optimization, and weak quality control in keyword selection. These issues hinder the agentic use of LLMs in fully automating keyword decisions by monitoring and reasoning over key performance indicators such as impressions, clicks, conversions, and CTA effectiveness. To overcome these challenges, we propose OMS, a keyword generation framework that is On-the-fly (requires no training data, monitors online performance, and adapts accordingly), Multi-objective (employs agentic reasoning to optimize keywords based on multiple performance metrics), and Self-reflective (agentically evaluates keyword quality). Experiments on benchmarks and real-world ad campaigns show that OMS outperforms existing methods; ablation and human evaluations confirm the effectiveness of each component and the quality of generated keywords.

Problem

• Automated keyword generation lacks multi-objective performance optimization
• Existing methods require large-scale query-keyword pair data
• Weak quality control in keyword selection hinders ad campaign success

Innovation

• On-the-fly keyword generation without training data
• Multi-objective optimization using agentic reasoning
• Self-reflective quality evaluation of keywords