CREST: Improving Interpretability and Effectiveness of Troubleshooting at Ericsson through Criterion-Specific Trouble Report Retrieval

2025-11-21

AI Summary

To address the low retrieval efficiency caused by the large volume and heterogeneous features of Telecom Reports (TRs), this paper proposes CREST, a two-stage multi-criteria decomposition retrieval framework. CREST first decomposes the multidimensional fault characteristics of TRs into several criterion-specific subtasks, each modeled by a dedicated Transformer-based retriever; fine-grained relevance scores are then ensembled via meta-learning. In the second stage, initial retrieval results are refined through re-ranking to enhance precision. Crucially, CREST introduces criterion-specific modeling to improve both interpretability and decision transparency. Evaluated on Ericsson’s real-world industrial TR dataset, CREST significantly outperforms single-model baselines, achieving substantial improvements in key metrics—including +12.7% in MRR and +9.3% in NDCG@10—thereby enabling rapid fault localization and supporting efficient software maintenance.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Feature Construction/Reformulation
• Computer Vision: Image and Video Retrieval

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

The rapid evolution of the telecommunication industry necessitates efficient troubleshooting processes to maintain network reliability, software maintainability, and service quality. Trouble Reports (TRs), which document issues in Ericsson's production system, play a critical role in facilitating the timely resolution of software faults. However, the complexity and volume of TR data, along with the presence of diverse criteria that reflect different aspects of each fault, present challenges for retrieval systems. Building on prior work at Ericsson, which utilized a two-stage workflow, comprising Initial Retrieval (IR) and Re-Ranking (RR) stages, this study investigates different TR observation criteria and their impact on the performance of retrieval models. We propose extbf{CREST} ( extbf{C}riteria-specific extbf{R}etrieval via extbf{E}nsemble of extbf{S}pecialized extbf{T}R models), a criterion-driven retrieval approach that leverages specialized models for different TR fields to improve both effectiveness and interpretability, thereby enabling quicker fault resolution and supporting software maintenance. CREST utilizes specialized models trained on specific TR criteria and aggregates their outputs to capture diverse and complementary signals. This approach leads to enhanced retrieval accuracy, better calibration of predicted scores, and improved interpretability by providing relevance scores for each criterion, helping users understand why specific TRs were retrieved. Using a subset of Ericsson's internal TRs, this research demonstrates that criterion-specific models significantly outperform a single model approach across key evaluation metrics. This highlights the importance of all targeted criteria used in this study for optimizing the performance of retrieval systems.

Problem

• Improves troubleshooting efficiency for telecom network reliability
• Addresses complexity of multi-criteria trouble report retrieval systems
• Enhances interpretability through criterion-specific specialized model ensembles

Innovation

• Criterion-driven retrieval approach using specialized models
• Ensemble aggregation of models for complementary signals
• Enhanced interpretability through criterion-specific relevance scores