Towards a scalable AI-driven framework for data-independent Cyber Threat Intelligence Information Extraction

2025-01-08

AI Summary

Existing cyber threat intelligence (CTI) extraction methods face challenges in efficiently and automatically performing joint entity-relation extraction due to fragmented, heterogeneous multi-source texts and severe scarcity of labeled data. This paper introduces 0-CTI, the first modular CTI extraction framework supporting both supervised and zero-shot learning, enabling end-to-end STIX-aligned information extraction without manual annotation. Its key contributions are: (1) a domain-agnostic architecture that decouples extraction tasks from domain-specific labeling requirements; (2) the first zero-shot approach capable of joint entity and relation extraction; and (3) adaptive performance across low-resource and high-resource scenarios. Built upon Transformer-based models and zero-shot NLP techniques, 0-CTI operates out-of-the-box in zero-shot mode and surpasses state-of-the-art supervised entity extraction methods. Crucially, all outputs strictly conform to the STIX 2.1 standard, significantly enhancing cross-organizational threat collaboration and response efficiency.

Technology Category

Natural Language Processing: Information ExtractionComputer Vision: Multi-modal VisionConstraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphsSearch and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAGWeb Mining and Content Analysis: Large pretrained models with web data

Abstract

Cyber Threat Intelligence (CTI) is critical for mitigating threats to organizations, governments, and institutions, yet the necessary data are often dispersed across diverse formats. AI-driven solutions for CTI Information Extraction (IE) typically depend on high-quality, annotated data, which are not always available. This paper introduces 0-CTI, a scalable AI-based framework designed for efficient CTI Information Extraction. Leveraging advanced Natural Language Processing (NLP) techniques, particularly Transformer-based architectures, the proposed system processes complete text sequences of CTI reports to extract a cyber ontology of named entities and their relationships. Our contribution is the development of 0-CTI, the first modular framework for CTI Information Extraction that supports both supervised and zero-shot learning. Unlike existing state-of-the-art models that rely heavily on annotated datasets, our system enables fully dataless operation through zero-shot methods for both Entity and Relation Extraction, making it adaptable to various data availability scenarios. Additionally, our supervised Entity Extractor surpasses current state-of-the-art performance in cyber Entity Extraction, highlighting the dual strength of the framework in both low-resource and data-rich environments. By aligning the system's outputs with the Structured Threat Information Expression (STIX) format, a standard for information exchange in the cybersecurity domain, 0-CTI standardizes extracted knowledge, enhancing communication and collaboration in cybersecurity operations.

Problem

• Network Threat Intelligence
• Unsupervised Learning
• Data Aggregation

Innovation

• 0-CTI AI Framework
• Unsupervised Learning
• STIX Format Integration