When to Trust the Map: Confidence-Aware LLM Routing for Automotive CVE-to-ATM Mapping

2026-07-21

AI Summary

This study addresses the challenge of erroneous automated mappings between automotive CVE vulnerability descriptions and the Automotive Threat Matrix (ATM) due to semantic inconsistencies. To mitigate this issue, the authors propose a confidence-aware selective automation approach that integrates hierarchical in-context learning to generate candidate mappings and combines self-consistency analysis with evidence verification signals from large language models. These signals are fused into a calibrated meta-model that outputs confidence scores, dynamically routing each mapping decision to one of three categories: AUTO, REVIEW, or HOLD. Experimental results demonstrate that under high-confidence conditions, the method achieves a precision of 0.878 for AUTO mappings—more than double that of the baseline—and attains a calibrated confidence AUROC of 0.868, substantially outperforming a flat zero-shot GPT-5.2 baseline and thereby enhancing decision reliability in safety-critical contexts.

Abstract

Public CVE descriptions report the technical conditions and impact of vulnerabilities, whereas the Auto-ISAC Automotive Threat Matrix (ATM) expresses an adversary's tactics and techniques. Because the two representations are not directly aligned, incorrect automated mappings in safety-critical environments may distort threat interpretation and mitigation prioritization, motivating a confidence-aware approach that distinguishes auto-confirmable mappings from uncertain cases. This paper reformulates automotive CVE-to-ATM mapping as a selective automation problem. The proposed framework generates candidate mappings via hierarchical in-context learning, then fuses self-consistency and LLM-based evidence verification signals into a calibrated meta-model. The resulting calibrated confidence score routes each candidate into AUTO, REVIEW, or HOLD. On the evaluation set, the proposed system substantially improved candidate-set precision at matched recall over a Flat zero-shot GPT-5.2 baseline. In the High-Confidence operating mode, the AUTO tier achieved a precision of 0.878, more than double the candidate-set base rate, and the calibrated confidence score achieved an AUROC of 0.868 in distinguishing correct from incorrect candidates. These results show that the framework can support selective automation by isolating auto-confirmable mappings from those requiring analyst review.

Problem

• CVE-to-ATM mapping
• confidence-aware routing
• selective automation
• threat intelligence alignment
• automotive cybersecurity

Innovation

• confidence-aware routing
• selective automation
• CVE-to-ATM mapping
• hierarchical in-context learning
• LLM-based verification