SoK: A Large-Scale Empirical Study of Emulation-Based Dynamic Analysis Research for ARM Cortex-M Firmware (Extended Version)

2026-09-29

AI Summary

This study addresses the limitations of existing evaluation datasets for firmware analysis tools and the lack of a holistic understanding of component interactions. To this end, it constructs the first unified automated analysis pipeline, leveraging the large-scale OTACAP and FirmLine datasets to systematically evaluate the collaborative mechanisms and stage-wise output support capabilities of 24 emulation-based analysis tools. By integrating fuzzing, code coverage analysis, and error diagnosis techniques, the investigation reveals that only 34.5% of samples can be successfully fuzzed, with generally low code coverage achieved. These findings expose fundamental deficiencies in current methodologies, providing empirical evidence and actionable directions for optimizing firmware analysis techniques.

Abstract

Microcontroller (MCU)-based devices are increasingly pervasive, making efficient, scalable firmware security analysis critical. Recent firmware re-hosting work enables automated vulnerability assessment, yet two gaps remain. First, existing tools are evaluated on limited, heavily overlapping datasets, undermining the generalizability of reported results. Second, prior research advances the state of the art along isolated dimensions, such as emulation, fuzzing, or bug diagnosis, without a holistic understanding of how these components interact and complement one another in dynamic analysis workflows. Building on recently released large-scale MCU firmware datasets from OTACAP and FirmLine, we present an empirical study of 24 emulation-based firmware analysis tools published in top conferences and journals. We position these tools within a unified automated analysis pipeline: emulation configuration reconnaissance, emulation, bug finding, and diagnosis. At each stage, we evaluate whether a tool's output provides sufficient information to enable the subsequent stage, using a deduplicated and validated subset of 4,571 ARM-based firmware samples. Only 1,580 samples (34.5%) can be successfully fuzzed, even counting a sample successful if at least one existing tool can fuzz it. Among these, fuzzing results are generally poor, with average code coverage of only 10% and many false crashes/hangs. Through a systematic analysis of failed fuzzing attempts and false-positive cases, we expose fundamental challenges and methodological limitations in current approaches. These findings highlight critical gaps in the state of the art and provide actionable insights to guide future research in emulation-based firmware analysis.

Problem

• firmware security analysis
• ARM Cortex-M
• emulation-based dynamic analysis
• fuzzing
• empirical study

Innovation

• Emulation-based Dynamic Analysis
• ARM Cortex-M Firmware
• Large-Scale Empirical Study
• Automated Analysis Pipeline
• Fuzzing