Natural Language Processing and Deep Learning Models to Classify Phase of Flight in Aviation Safety Occurrences

2023-09-06

2023 IEEE Region 10 Symposium (TENSYMP)

AI Summary

This study addresses the low efficiency and poor annotation consistency in manual flight-phase labeling (seven categories) for aviation safety analysis. We propose an automated classification method for unstructured accident report texts. Methodologically, we introduce simplified Recurrent Neural Networks (sRNNs) and simplified ResNets—novel applications to multi-phase aviation safety text classification—integrated with NLP preprocessing and a multi-class supervised learning framework. Evaluated on 27,000 real-world NTSB reports, the sRNN achieves >68% accuracy, outperforming both the simplified ResNet and a random baseline (14%) across precision, recall, and F1-score, thereby demonstrating its superior capability in modeling temporal semantics. This work provides a scalable, robust automation solution for aviation risk root-cause analysis and phase-specific safety assessment.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

This study focuses on the classification of safety occurrences in the air transport system using natural language processing (NLP) and artificial intelligence (AI) models. The researchers utilized ResNet and sRNN deep learning models to classify flight phases based on unstructured text narratives of safety occurrence reports from the NTSB. The study evaluated the performance of these models using a dataset of 27,000 safety occurrence reports and found that both models achieved an accuracy exceeding 68%, surpassing the random guess rate of 14% for the seven-class classification problem. Additionally, the models exhibited high precision, recall, and F1 scores. Notably, the sRNN model outperformed the simplified ResNet model architecture used in the study. These findings suggest that NLP and deep learning models can effectively extract flight phase information from raw text narratives, enabling the thorough analysis of safety occurrences in the aviation industry.

Problem

• Automated Classification
• Aircraft Accident Reports
• Flight Phases

Innovation

• NLP
• Deep Learning
• sRNN