PushGen: Push Notifications Generation with LLM

2025-12-16

AI Summary

To address low user engagement caused by uncontrollable stylistic outputs and poor quality assessment in LLM-based push notification generation, this paper proposes an industrial-scale automated framework. First, it introduces controllable category prompting to enable fine-grained stylistic guidance. Second, it develops a lightweight learnable reward model to rank and filter candidate notifications based on quality. Third, it establishes a closed-loop optimization pipeline integrating offline evaluation (yielding significant improvements in BLEU and ROUGE scores) and online A/B testing. This work pioneers a high-quality, style-controllable, and quantitatively evaluable push notification generation paradigm. Deployed in a large-scale production system, the framework serves over 100 million users daily and achieves an average 12.7% lift in click-through rate (CTR).

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We present PushGen, an automated framework for generating high-quality push notifications comparable to human-crafted content. With the rise of generative models, there is growing interest in leveraging LLMs for push content generation. Although LLMs make content generation straightforward and cost-effective, maintaining stylistic control and reliable quality assessment remains challenging, as both directly impact user engagement. To address these issues, PushGen combines two key components: (1) a controllable category prompt technique to guide LLM outputs toward desired styles, and (2) a reward model that ranks and selects generated candidates. Extensive offline and online experiments demonstrate its effectiveness, which has been deployed in large-scale industrial applications, serving hundreds of millions of users daily.

Problem

• Automates push notification generation using LLMs
• Ensures stylistic control and quality assessment
• Enhances user engagement through ranking and selection

Innovation

• Controllable category prompts guide LLM style
• Reward model ranks and selects generated candidates
• Deployed at scale serving millions daily