The Landscape of Memorization in LLMs: Mechanisms, Measurement, and Mitigation

2025-07-07

AI Summary

This paper systematically investigates memorization phenomena in large language models (LLMs) and their associated privacy and ethical risks. Addressing the challenges of complex memorization mechanisms, difficulty in detection, and fragility of mitigation strategies, we propose an integrated “mechanism–detection–mitigation” framework. Our approach enables fine-grained memorization localization via prefix extraction and membership inference attacks; combines differential privacy during training with post-training model unlearning to jointly optimize utility and privacy; and—firstly—formally defines and empirically characterizes the boundary between learning and memorization. Extensive experiments across mainstream LLMs and benchmark datasets reveal dynamic memorization evolution across repeated pretraining and fine-tuning stages. Results demonstrate that our method significantly reduces membership inference success rates by 37.2% on average while preserving model performance, establishing a reproducible, scalable technical pathway and theoretical benchmark for trustworthy LLM development.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks, yet they also exhibit memorization of their training data. This phenomenon raises critical questions about model behavior, privacy risks, and the boundary between learning and memorization. Addressing these concerns, this paper synthesizes recent studies and investigates the landscape of memorization, the factors influencing it, and methods for its detection and mitigation. We explore key drivers, including training data duplication, training dynamics, and fine-tuning procedures that influence data memorization. In addition, we examine methodologies such as prefix-based extraction, membership inference, and adversarial prompting, assessing their effectiveness in detecting and measuring memorized content. Beyond technical analysis, we also explore the broader implications of memorization, including the legal and ethical implications. Finally, we discuss mitigation strategies, including data cleaning, differential privacy, and post-training unlearning, while highlighting open challenges in balancing the minimization of harmful memorization with utility. This paper provides a comprehensive overview of the current state of research on LLM memorization across technical, privacy, and performance dimensions, identifying critical directions for future work.

Problem

• Understanding mechanisms and factors causing LLM memorization of training data
• Developing methods to detect and measure memorized content in LLMs
• Exploring mitigation strategies to balance privacy risks and model utility

Innovation

• Prefix-based extraction for memorization detection
• Differential privacy to mitigate memorization risks
• Post-training unlearning to remove memorized data