LAG-XAI: A Lie-Inspired Affine Geometric Framework for Interpretable Paraphrasing in Transformer Latent Spaces

2026-04-07

AI Summary

This work addresses the limited geometric interpretability of internal semantic representations in Transformer language models. It introduces, for the first time, a Lie group and affine geometry perspective to model paraphrasing as structured affine transformations on a semantic manifold. By leveraging mean-field approximation, these transformations are decomposed into interpretable components—rotation, translation, and deformation—revealing local isometry in semantic space and a stable reconstruction angle of approximately 27.84°. The framework further enables hallucination detection through geometric deviation analysis, achieving an AUC of 0.7713 on PIT-2015 (roughly 80% of the nonlinear baseline performance) and automatically identifying 95.3% of factual distortions in HaluEval via geometric consistency checks.

Technology Category

Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal ReasoningMachine Learning: Large Multimodal Models (LMMs)

Application Category

Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsSearch and Retrieval-Augmented AI: Large language models for searchUser Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Modern Transformer-based language models achieve strong performance in natural language processing tasks, yet their latent semantic spaces remain largely uninterpretable black boxes. This paper introduces LAG-XAI (Lie Affine Geometry for Explainable AI), a novel geometric framework that models paraphrasing not as discrete word substitutions, but as a structured affine transformation within the embedding space. By conceptualizing paraphrasing as a continuous geometric flow on a semantic manifold, we propose a computationally efficient mean-field approximation, inspired by local Lie group actions. This allows us to decompose paraphrase transitions into geometrically interpretable components: rotation, deformation, and translation. Experiments on the noisy PIT-2015 Twitter corpus, encoded with Sentence-BERT, reveal a "linear transparency" phenomenon. The proposed affine operator achieves an AUC of 0.7713. By normalizing against random chance (AUC 0.5), the model captures approximately 80% of the non-linear baseline's effective classification capacity (AUC 0.8405), offering explicit parametric interpretability in exchange for a marginal drop in absolute accuracy. The model identifies fundamental geometric invariants, including a stable matrix reconfiguration angle (~27.84°) and near-zero deformation, indicating local isometry. Cross-domain generalization is confirmed via direct cross-corpus validation on an independent TURL dataset. Furthermore, the practical utility of LAG-XAI is demonstrated in LLM hallucination detection: using a "cheap geometric check," the model automatically detected 95.3% of factual distortions on the HaluEval dataset by registering deviations beyond the permissible semantic corridor. This approach provides a mathematically grounded, resource-efficient path toward the mechanistic interpretability of Transformers.

Problem

• interpretability
• paraphrasing
• latent space
• Transformer
• semantic manifold

Innovation

• affine geometry
• Lie group
• interpretable paraphrasing
• semantic manifold
• mechanistic interpretability