Syntax-Guided Diffusion Language Models with User-Integrated Personalization

2025-10-01

AI Summary

Large language models often produce generic text lacking structural diversity and personalized expression. To address this, we propose Syntax-Guided Diffusion Language Models (SG-DLMs), which jointly optimize text generation via syntactic structure supervision and personalized conditional control. Methodologically: (1) we design cascaded and non-cascaded dual architectures to decouple and jointly model syntax and semantics; (2) we introduce a shared latent representation mechanism enabling cross-user personalization and zero-shot style transfer; (3) we integrate structured syntactic priors with conditional diffusion to achieve fine-grained, interpretable stylistic control. Experiments demonstrate that SG-DLM significantly outperforms baselines in fluency, lexical and structural diversity, and style fidelity. Moreover, it exhibits superior controllability and generalization in personalized text generation tasks, particularly under low-resource and unseen-style settings.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Diffusion Models for Vision

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large language models have made revolutionary progress in generating human-like text, yet their outputs often tend to be generic, exhibiting insufficient structural diversity, which limits personalized expression. Recent advances in diffusion models have opened new opportunities for improving language generation beyond the limitations of autoregressive paradigms. In this work, we propose a syntax-guided diffusion language model that integrates structural supervision and personalized conditioning to enhance text quality, diversity, and controllability. We introduce a cascaded framework that generates syntactic guidance before conditional text generation, and further generalize it to a novel noncascaded architecture for better alignment between structure and content. By incorporating syntactic information in the generating process, the proposed model better captures the lexical and structural characteristics of stylistic sentence construction. To enable fine-grained personalization, we develop a shared representation mechanism that facilitates information integration across users, supporting both faithful stylistic generation and generalizable zero-shot inference. Extensive experiments on multiple tasks demonstrate the superiority of our approach in fluency, diversity, and stylistic fidelity. Further qualitative analyses highlight its interpretability and flexibility in learning personalized patterns.

Problem

• Enhancing text diversity and personalization in language models
• Improving structural alignment between syntax and generated content
• Enabling fine-grained stylistic control through shared user representations

Innovation

• Syntax-guided diffusion model enhances text diversity
• Cascaded framework generates syntactic guidance first
• Shared representation enables fine-grained personalization integration