Transparent AI for Mathematics: Transformer-Based Large Language Models for Mathematical Entity Relationship Extraction with XAI

2026-03-06

AI Summary

This study addresses the challenge of automatically parsing specialized entities and their complex relationships in mathematical texts. It formalizes this problem as a Mathematical Entity Relation Extraction (MERE) task and proposes a BERT-based Transformer model to automatically identify operands (entities) and operators (relations). To enhance model interpretability, the approach integrates SHAP (SHapley Additive exPlanations) for feature importance analysis. Evaluated on a newly constructed dataset, the method achieves a relation extraction accuracy of 99.39%, while simultaneously providing transparent insights into its decision-making process through interpretable feature attributions. By balancing high precision with explainability, this work offers a robust foundation for applications in intelligent tutoring systems and the construction of mathematical knowledge graphs.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Mathematical text understanding is a challenging task due to the presence of specialized entities and complex relationships between them. This study formulates mathematical problem interpretation as a Mathematical Entity Relation Extraction (MERE) task, where operands are treated as entities and operators as their relationships. Transformer-based models are applied to automatically extract these relations from mathematical text, with Bidirectional Encoder Representations from Transformers (BERT) achieving the best performance, reaching an accuracy of 99.39%. To enhance transparency and trust in the model's predictions, Explainable Artificial Intelligence (XAI) is incorporated using Shapley Additive Explanations (SHAP). The explainability analysis reveals how specific textual and mathematical features influence relation prediction, providing insights into feature importance and model behavior. By combining transformer-based learning, a task-specific dataset, and explainable modeling, this work offers an effective and interpretable framework for MERE, supporting future applications in automated problem solving, knowledge graph construction, and intelligent educational systems.

Problem

• Mathematical Entity Relation Extraction
• Mathematical Text Understanding
• Explainable AI
• Transformer Models
• Entity Relationship Extraction

Innovation

• Mathematical Entity Relation Extraction
• Transformer-based LLMs
• Explainable AI (XAI)
• SHAP
• BERT