Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing

2026-10-01

AI Summary

This study addresses the context dependency inherent in unstructured knowledge editing, wherein models can reproduce edited passages yet fail to reliably recall independent atomic facts. To this end, this work proposes FOVEATED, a framework that exposes and rectifies the context-induced underestimation of editing difficulty. Specifically, it introduces a plug-and-play foveated view mechanism that perturbs contextual key-value positions during the editing phase via randomly shifted RoPE positional encodings, while restoring native encodings at inference time, thereby enhancing the memorization of standalone facts. Extensive experiments conducted across five editors, two large language model backbones, and three benchmarks demonstrate that the proposed approach consistently and significantly improves atomic fact recall performance.

Abstract

Large language models (LLMs) increasingly serve as general-purpose interfaces to factual knowledge, but their parameters do not automatically reflect information that changes after pretraining. Knowledge editing (KE) provides a targeted alternative to costly retraining by modifying selected knowledge and preserving unrelated knowledge and general capabilities. Conventional KE uses structured factual triples, whereas unstructured KE (UKE) uses free-form passages containing multiple facts. Nonetheless, existing UKE editors exhibit a failure mode known as context reliance: edited LLMs can often reproduce the editing passage but fail to reliably recall its individual facts without the original passage context. We identify context-induced difficulty underestimation under the standard passage-level editing objective: later facts receive increasingly rich ground-truth context and consequently incur lower initial losses, making them appear easier to learn. In response, we propose FOVEATED, a plug-and-play framework that constructs focused views of each sentence by randomly shifting the Rotary Position Embedding (RoPE) positions assigned to the keys of its preceding context. The perturbation is applied during editing and removed afterward, leaving the model's native positional encoding unchanged at inference time. We instantiate FOVEATED for both direct-optimization and locate-then-edit editors. We theoretically analyze how FOVEATED counteracts context-induced difficulty underestimation and empirically demonstrate consistent improvements across five KE editors, two LLM backbones, and three benchmarks.

Problem

• Unstructured Knowledge Editing
• Atomic-Fact Recall
• Context Reliance
• Large Language Models

Innovation

• Unstructured Knowledge Editing
• Atomic-Fact Recall
• Context Reliance
• Rotary Position Embedding (RoPE)
• Focused Views