msf-CNN: Patch-based Multi-Stage Fusion with Convolutional Neural Networks for TinyML

2025-05-16

AI Summary

Deploying CNNs on microcontrollers (MCUs) faces dual constraints of extremely limited memory (e.g., 128 KB RAM) and stringent real-time latency requirements. Method: This paper proposes an efficient TinyML deployment framework featuring a multi-stage patch fusion space search algorithm based on directed acyclic graph (DAG) traversal, systematically expanding the exploration of optimal layer-fusion configurations. It integrates patch-based inter-layer fusion with dataflow optimization to enable cross-architecture deployment on ARM Cortex-M, RISC-V, and ESP32 platforms. Contribution/Results: Experiments demonstrate a 50% reduction in inference memory footprint compared to MCUNetV2 and StreamNet, achieving real-time, low-RAM CNN execution across diverse MCUs. The approach significantly enhances both execution efficiency and design flexibility for resource-constrained edge devices.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

AI spans from large language models to tiny models running on microcontrollers (MCUs). Extremely memory-efficient model architectures are decisive to fit within an MCU's tiny memory budget e.g., 128kB of RAM. However, inference latency must remain small to fit real-time constraints. An approach to tackle this is patch-based fusion, which aims to optimize data flows across neural network layers. In this paper, we introduce msf-CNN, a novel technique that efficiently finds optimal fusion settings for convolutional neural networks (CNNs) by walking through the fusion solution space represented as a directed acyclic graph. Compared to previous work on CNN fusion for MCUs, msf-CNN identifies a wider set of solutions. We published an implementation of msf-CNN running on various microcontrollers (ARM Cortex-M, RISC-V, ESP32). We show that msf-CNN can achieve inference using 50% less RAM compared to the prior art (MCUNetV2 and StreamNet). We thus demonstrate how msf-CNN offers additional flexibility for system designers.

Problem

• Optimize memory usage for CNNs on microcontrollers
• Reduce inference latency in real-time AI applications
• Enhance fusion solution space for efficient data flow

Innovation

• Patch-based fusion optimizes CNN data flows
• Directed acyclic graph represents fusion solutions
• 50% less RAM usage than prior art