When Memory Becomes Authority: Benchmarking Authority Collapse at the Memory Consolidation Boundary

2026-08-03

AI Summary

This work addresses the "authority collapse" problem in large language model (LLM) agents, wherein memory integration mechanisms often disregard source-specific permission constraints, leading agents to erroneously treat retrieved memories as high-authority knowledge. The study formally defines and quantifies this issue, introducing AuthMem-Bench—a controlled paired benchmark—to systematically evaluate the authority preservation capabilities of seven memory integrators and seven LLMs across 49 configurations. Experimental results reveal authority collapse in 48 of these configurations. Crucially, incorporating memory mechanisms augmented with authority metadata reduces unauthorized behavior from 16.9% to 0.0% without compromising task success rates, demonstrating the efficacy of such metadata in preventing unauthorized knowledge usage.

Abstract

Persistent memory allows (self-evolving) LLM agents to adapt across tasks by consolidating heterogeneous interaction histories into reusable facts, preferences, observations, and rules. Yet consolidation also imposes an implicit authorization boundary: it determines whether stored information may later be consumed as a user fact, an attested observation, or a standing instruction. We identify authority collapse, in which consolidation preserves a claim while erasing the source constraints governing its authorized use, causing the stored memory to imply greater authority than its source permits. We introduce AuthMem-Bench, a controlled paired benchmark that holds the focal claim and downstream task fixed while varying only source authority. It evaluates write-time collapse, downstream authorization errors, and automatic authority preservation. Across seven consolidators based on widely used agent-memory systems and seven LLM backbones, we observe authority collapse in 48 of 49 evaluated configurations. In a controlled action-grounded evaluation, collapsed memories without authority metadata yield a mean unauthorized-action rate of 50.3%. In an end-to-end evaluation, automatically predicted and persisted authority labels reduce the observed unauthorized-action rate from 16.9% to 0.0%, while benign task success remains essentially unchanged. These findings show that memory-driven adaptation must preserve not only what was learned, but also the authority under which it may be reused.

Problem

• authority collapse
• memory consolidation
• LLM agents
• authorization boundary
• persistent memory

Innovation

• authority collapse
• memory consolidation
• LLM agents
• AuthMem-Bench
• authorization boundary