The Application of Transformer-Based Models for Predicting Consequences of Cyber Attacks

2025-08-18

AI Summary

This work addresses the critical security modeling task of predicting cyberattack consequences. We propose a multi-label classification method integrating BERT with a Hierarchical Attention Network (HAN) to automatically parse textual descriptions of vulnerabilities from the MITRE CWE database, enabling fine-grained identification of impacts across dimensions including availability, confidentiality, integrity, and access control. To our knowledge, this is the first study to jointly leverage pretrained language models and hierarchical attention for cybersecurity impact prediction, effectively capturing both semantic attack features and label-level hierarchical relationships. Experimental results demonstrate that the BERT-based model achieves an overall accuracy of 0.972—significantly outperforming CNN- and LSTM-based baselines—while HAN exhibits superior recall on sparse labels. This work advances the automation of threat impact analysis and empirically validates the effectiveness and generalizability of Transformer-based architectures for cybersecurity text understanding.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Natural Language Processing: Safety and Robustness

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Cyberattacks are increasing, and securing against such threats is costing industries billions of dollars annually. Threat Modeling, that is, comprehending the consequences of these attacks, can provide critical support to cybersecurity professionals, enabling them to take timely action and allocate resources that could be used elsewhere. Cybersecurity is heavily dependent on threat modeling, as it assists security experts in assessing and mitigating risks related to identifying vulnerabilities and threats. Recently, there has been a pressing need for automated methods to assess attack descriptions and forecast the future consequences of the increasing complexity of cyberattacks. This study examines how Natural Language Processing (NLP) and deep learning can be applied to analyze the potential impact of cyberattacks by leveraging textual descriptions from the MITRE Common Weakness Enumeration (CWE) database. We emphasize classifying attack consequences into five principal categories: Availability, Access Control, Confidentiality, Integrity, and Other. This paper investigates the use of Bidirectional Encoder Representations from Transformers (BERT) in combination with Hierarchical Attention Networks (HANs) for Multi-label classification, evaluating their performance in comparison with conventional CNN and LSTM-based models. Experimental findings show that BERT achieves an overall accuracy of $0.972$, far higher than conventional deep learning models in multi-label classification. HAN outperforms baseline forms of CNN and LSTM-based models on specific cybersecurity labels. However, BERT consistently achieves better precision and recall, making it more suitable for predicting the consequences of a cyberattack.

Problem

• Predicting cyber attack consequences using Transformer models
• Automating threat modeling with NLP and deep learning
• Classifying attack impacts into five key categories

Innovation

• Transformer-based BERT for cyberattack consequence prediction
• Hierarchical Attention Networks for multi-label classification
• NLP analysis of MITRE CWE textual descriptions