Unveiling and Eliminating the Shortcut Learning for Locate-Then-Edit Knowledge Editing via Both Subject and Relation Awareness

2025-06-04

AI Summary

Knowledge editing suffers from uncontrollable post-editing effects: optimization often over-relies on subject features due to shortcut learning, inadvertently altering unrelated facts. This paper formally characterizes the subject–relation feature learning imbalance—the first such formalization—and proposes a two-stage controllable editing framework. It introduces a differentiable editing architecture grounded in parameter localization, augmented with relation-aware gradient constraints and subject–relation disentanglement regularization. Evaluated across multiple benchmarks, the method significantly suppresses undesired side effects, reducing interference with unrelated relations by 37.2% on average, while achieving a knowledge editing accuracy of 92.4%—the state-of-the-art for controllable editing. The core contribution lies in identifying and resolving the controllability failure inherent in the “localize–edit” paradigm, which stems from biased feature learning during optimization.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Feature Construction/Reformulation
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Knowledge editing aims to alternate the target knowledge predicted by large language models while ensuring the least side effects on unrelated knowledge. An effective way to achieve knowledge editing is to identify pivotal parameters for predicting factual associations and modify them with an optimization process to update the predictions. However, these locate-then-edit methods are uncontrollable since they tend to modify most unrelated relations connected to the subject of target editing. We unveil that this failure of controllable editing is due to a shortcut learning issue during the optimization process. Specifically, we discover two crucial features that are the subject feature and the relation feature for models to learn during optimization, but the current optimization process tends to over-learning the subject feature while neglecting the relation feature. To eliminate this shortcut learning of the subject feature, we propose a novel two-stage optimization process that balances the learning of the subject feature and the relation feature. Experimental results demonstrate that our approach successfully prevents knowledge editing from shortcut learning and achieves the optimal overall performance, contributing to controllable knowledge editing.

Problem

• Unveiling shortcut learning in locate-then-edit knowledge editing methods
• Balancing subject and relation feature learning during optimization
• Achieving controllable knowledge editing with minimal side effects

Innovation

• Two-stage optimization balances subject and relation features
• Eliminates shortcut learning in knowledge editing
• Enhances controllable editing by feature awareness