MASE: Interpretable NLP Models via Model-Agnostic Saliency Estimation

2025-12-03

AI Summary

Insufficient interpretability of NLP models—particularly the lack of reliable, model-agnostic local explanations for discrete text inputs—hinders trust and transparency. To address this, we propose MASE, a model-agnostic explanation framework that estimates input feature importance by applying normalized linear Gaussian perturbations (NLGP) directly to the word embedding layer, thereby avoiding unstable discrete token perturbations. This design enhances explanation stability and robustness without requiring access to model internals or architectural assumptions. MASE is fully compatible with arbitrary black-box text classifiers. Extensive evaluation demonstrates that MASE significantly outperforms state-of-the-art model-agnostic methods—including LIME and SHAP—on metrics such as Delta Accuracy. By improving the traceability of prediction rationales and revealing model behavior more faithfully, MASE establishes a novel paradigm for trustworthy, post-hoc explanation in NLP.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Machine Learning: Transparent, Interpretable, Explainable ML
• Computer Vision: Interpretability, Explainability, and Transparency

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Deep neural networks (DNNs) have made significant strides in Natural Language Processing (NLP), yet their interpretability remains elusive, particularly when evaluating their intricate decision-making processes. Traditional methods often rely on post-hoc interpretations, such as saliency maps or feature visualization, which might not be directly applicable to the discrete nature of word data in NLP. Addressing this, we introduce the Model-agnostic Saliency Estimation (MASE) framework. MASE offers local explanations for text-based predictive models without necessitating in-depth knowledge of a model's internal architecture. By leveraging Normalized Linear Gaussian Perturbations (NLGP) on the embedding layer instead of raw word inputs, MASE efficiently estimates input saliency. Our results indicate MASE's superiority over other model-agnostic interpretation methods, especially in terms of Delta Accuracy, positioning it as a promising tool for elucidating the operations of text-based models in NLP.

Problem

• Interpretability of deep neural networks in NLP
• Model-agnostic saliency estimation for text models
• Local explanations without internal architecture knowledge

Innovation

• Model-agnostic saliency estimation framework for NLP
• Uses normalized linear Gaussian perturbations on embeddings
• Provides local explanations without model architecture knowledge