All On-Board: Fully On-Chip Neuromorphic Q-Learning with Embedded CartPole Simulation

2026-09-26

AI Summary

This study addresses the high energy consumption of AI models and the challenges of deploying reinforcement learning in resource-constrained environments by proposing the first fully embedded neuromorphic reinforcement learning agent. Built upon the Loihi 2 neuromorphic chip, the approach leverages spiking neural networks and on-chip environment emulation to achieve entirely on-chip closed-loop Q-learning without external host intervention, as demonstrated on the integrated CartPole task. Experimental results indicate that the proposed system reduces dynamic power consumption by two orders of magnitude and decreases execution time by 50%, while delivering control performance comparable to conventional CPU-based implementations. This work establishes an efficient hardware paradigm for low-power autonomous online learning at the edge.

Abstract

As AI models grow in size and usage, their energy demands increase dramatically, raising sustainability and economic concerns. Neuromorphic hardware, inspired by the energy efficiency of the brain, seeks to address this challenge by offering low-power, fast-processing alternatives to conventional computing. Such hardware is particularly well-suited to control systems deployed in resource-constrained environments, which are best trained via reinforcement learning (RL). This contribution presents the design and implementation of a fully on-chip, closed-loop Loihi 2 RL agent. Our neuromorphic circuit consists of a fully embedded Q-learning algorithm and an on-chip simulation of the CartPole-v0 environment on Loihi 2. Our Q-learning algorithm trained the same number of successful agents as the CPU implementation in only half the execution time and with two orders of magnitude less dynamic power. These findings demonstrate the viability of RL on neuromorphic hardware and highlight its promise for building energy-efficient, real-time, embedded AI systems.

Problem

• Neuromorphic hardware
• Reinforcement learning
• Q-learning
• Energy efficiency
• On-chip simulation

Innovation

• Neuromorphic hardware
• On-chip Q-learning
• Loihi 2
• Reinforcement learning
• Embedded AI