Melo: A Production LLM-Powered Music Recommendation Agent

2026-07-26

AI Summary

This work addresses the challenges of entity hallucination and degradation to generic popular recommendations in large language model (LLM)-based music recommendation systems, particularly for long-tail user requests. To mitigate these issues, the authors propose an LLM-driven recommendation framework grounded in a deterministic five-node state graph. The system employs a state machine to orchestrate heterogeneous tools, integrating runtime entity grounding and a session-level reflection-and-retry strategy to enable verified decision-making and recovery from failures without resorting to blind fallbacks. The approach combines a production-grade search index, state-machine-guided tool orchestration, and a three-layer entity grounding stack. Online A/B tests demonstrate a >2 percentage point increase in playlist retention and over one minute of additional core engagement duration. Offline evaluations show a 7.8 percentage point reduction in entity misidentification, with 5.8% of sessions triggering reflection-and-retry—59% of which successfully recovered.

Abstract

We describe Melo, an LLM-powered music recommendation agent deployed on NetEase Cloud Music. Melo is structured as a deterministic five-node state graph over heterogeneous tools, with a prompt- and state-machine-driven orchestration policy rather than a fine-tuned controller. At industrial scale, the bottleneck is not how smart the brain is but how the system detects and recovers from the mistakes that brain makes. Two production failure modes drove the design: entity hallucination, where the agent commits to interpretations unsupported by the live catalog or user-behavior index, and long-tail degradation, where over-constrained requests collapse to generic popular fallbacks. We address them with two complementary mechanisms. Inference-time entity grounding repurposes the production search index as a verification primitive that gates entity decisions before they propagate downstream. Reflective retry verbalizes failure reasons from a broken tool chain and feeds them into the next planning step, so the system can relax or revise constraints rather than fall back blindly. A one-month online A/B test across NetEase Cloud Music's playlist surfaces reports an over 2 pp lift in a primary playlist retention metric and a lift of over one minute in a core playlist engagement metric. Offline ablation isolates a 7.8 pp reduction in entity misidentification from the three-layer grounding stack on our evaluation set, and a triggered-session analysis on our evaluation set shows reflective retry firing on 5.8% of sessions with 59% process-level recovery. Our deployment experience suggests that progress on LLM-powered music recommendation at this scale depends as much on the named, ablatable runtime machinery that catches and corrects the brain's mistakes as on the brain itself: a hypothesis we offer for the community to test.

Problem

• entity hallucination
• long-tail degradation
• music recommendation
• LLM-powered agent
• production failure modes

Innovation

• LLM-powered recommendation
• entity grounding
• reflective retry
• state-machine orchestration
• music recommendation agent