SCDTour: Embedding Axis Ordering and Merging for Interpretable Semantic Change Detection

2025-09-15

AI Summary

In semantic change detection (SCD), there exists a fundamental trade-off between embedding interpretability and detection performance. To address this, we propose an axis-guided embedding compression framework that jointly models semantic similarity and change contribution, enabling ranking and hierarchical clustering of predefined semantic axes—thereby achieving substantial dimensionality reduction while preserving salient change signals. Our key contributions are: (1) a quantifiable evaluation mechanism for interpretable semantic axes; (2) an axis aggregation strategy balancing local discriminability and global consistency; and (3) co-optimization of high interpretability—driven explicitly by human-defined semantic axes—and strong performance, matching or surpassing full-dimensional embeddings across multiple benchmarks. Experimental results demonstrate that our method enables fine-grained characterization of lexical semantic evolution trajectories and significantly enhances model decision transparency.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Data Mining & Knowledge Management: Semantic Web
• Machine Learning: Dimensionality Reduction/Feature Selection

Application Category

• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

In Semantic Change Detection (SCD), it is a common problem to obtain embeddings that are both interpretable and high-performing. However, improving interpretability often leads to a loss in the SCD performance, and vice versa. To address this problem, we propose SCDTour, a method that orders and merges interpretable axes to alleviate the performance degradation of SCD. SCDTour considers both (a) semantic similarity between axes in the embedding space, as well as (b) the degree to which each axis contributes to semantic change. Experimental results show that SCDTour preserves performance in semantic change detection while maintaining high interpretability. Moreover, agglomerating the sorted axes produces a more refined set of word senses, which achieves comparable or improved performance against the original full-dimensional embeddings in the SCD task. These findings demonstrate that SCDTour effectively balances interpretability and SCD performance, enabling meaningful interpretation of semantic shifts through a small number of refined axes. Source code is available at https://github.com/LivNLP/svp-tour .

Problem

• Balancing interpretability and performance in semantic change detection
• Ordering and merging axes to reduce performance degradation
• Preserving detection accuracy while maintaining embedding interpretability

Innovation

• Orders and merges interpretable axes
• Considers semantic similarity and change contribution
• Produces refined word senses for performance