Reducing Instruction-Fetch Energy in RISC-V for Embedded AI Processing via Dynamic and Static Loop Caching

2026-07-24

AI Summary

This work addresses the high instruction fetch energy consumption in embedded RISC-V processors for edge AI inference, where SRAM-based instruction fetching accounts for over 40% of total energy. The authors propose the first RISC-V architecture integrating both dynamic and static loop caches: the dynamic cache automatically captures short backward-branch loops at runtime, while the static cache is software-managed and preloads hotspot code blocks at boot time. Evaluated on the open-source NEORV32 processor implemented in GlobalFoundries 22nm FDX+ technology at 0.5 V and 250 MHz using LeNet-5 inference as a benchmark, the dynamic loop cache reduces instruction fetch energy by 48.3% and total energy by 21.5%, whereas the static loop cache achieves an 83.3% reduction in instruction fetches and saves 35.5% in energy. Both caches incur area overheads of less than 0.2% of the total SoC area.

Abstract

Embedded RISC-V processors are increasingly deployed for on-device AI inference at the edge, where energy efficiency is a primary design constraint. Instruction fetching from SRAM-based memory is a dominant source of energy consumption in these cores, accounting for over 40\% of total energy in our baseline measurements. This paper presents two loop cache architectures integrated into the datapath of a RISC-V processor: a dynamic loop cache that automatically detects and caches short backward-branch loops at runtime, and a static loop cache that functions as a software-managed hot-code instruction buffer, allowing preloading of arbitrary instruction blocks during the boot sequence. Both designs are implemented in the open-source NEORV32 RISC-V processor and evaluated on a LeNet-5 convolutional neural network inference workload, synthesized on GlobalFoundries 22nm FDX+ technology at 0.5\,V and 250\,MHz. The dynamic cache reduces instruction fetches by 48.3\% and total energy by 21.5\%, while the static cache achieves an 83.3\% fetch reduction and 35.5\% total energy savings. The area overhead of both designs remains below 0.2\% of the full SoC area. The complete implementation is open source.

Problem

• instruction-fetch energy
• RISC-V
• embedded AI
• loop caching
• energy efficiency

Innovation

• loop cache
• instruction fetch energy
• RISC-V
• embedded AI
• energy efficiency