Comparative Study of Deep Learning Architectures for Textual Damage Level Classification

2024-03-21

SPIN

AI Summary

This study addresses the classification of aircraft damage severity levels from unstructured textual descriptions in aviation safety incident reports, aiming to support automated root-cause analysis and risk assessment. We propose a four-class deep learning framework and systematically compare LSTM, BiLSTM, GRU, and a simplified recurrent neural network (sRNN), all employing standardized NLP preprocessing and word embedding representations for end-to-end text classification. Experimental results show that all models achieve accuracy exceeding 88%, substantially outperforming a 25% random baseline; notably, the sRNN attains 89% accuracy and the highest recall, establishing a new performance benchmark. To our knowledge, this is the first work to empirically validate the efficacy of lightweight recurrent architectures for aviation damage text classification. The findings introduce an efficient, interpretable paradigm for domain-specific text classification under resource-constrained conditions.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Given the paramount importance of safety in the aviation industry, even minor operational anomalies can have significant consequences. Comprehensive documentation of incidents and accidents serves to identify root causes and propose safety measures. However, the unstructured nature of incident event narratives poses a challenge for computer systems to interpret. Our study aimed to leverage Natural Language Processing (NLP) and deep learning models to analyze these narratives and classify the aircraft damage level incurred during safety occurrences. Through the implementation of LSTM, BLSTM, GRU, and sRNN deep learning models, our research yielded promising results, with all models showcasing competitive performance, achieving an accuracy of over 88% significantly surpassing the 25% random guess threshold for a four-class classification problem. Notably, the sRNN model emerged as the top performer in terms of recall and accuracy, boasting a remarkable 89%. These findings underscore the potential of NLP and deep learning models in extracting actionable insights from unstructured text narratives, particularly in evaluating the extent of aircraft damage within the realm of aviation safety occurrences.

Problem

• Deep Learning Models
• Text Analysis
• Aircraft Damage Assessment

Innovation

• Deep Learning Models
• sRNN Optimality
• Aircraft Safety Assessment