Toward Self-Healing Networks-on-Chip: RL-Driven Routing in 2D Torus Architectures

2025-12-15

AI Summary

To address routing failures caused by node faults in 2D torus Network-on-Chip (NoC) topologies, this paper proposes an adaptive minimal routing method based on a multi-agent deep reinforcement learning framework using a Proximal Policy Optimization (PPO) variant. Each router is modeled as an autonomous, state-aware RL agent, enabling fully distributed, dynamic path selection—replacing conventional static shortest-path routing with fixed detour strategies. The approach is implemented within an extended BookSim simulation platform, supporting fine-grained fault injection and load-adaptive operation. Experimental results demonstrate that under 30–40 randomly injected node faults, the proposed method maintains a packet delivery rate exceeding 90% (compared to ~70% for baseline approaches), achieves a 20–30% improvement in throughput under high-load conditions, and significantly outperforms existing adaptive routing schemes in terms of network connectivity and fault recovery capability.

Technology Category

Planning, Routing, and Scheduling: Learning for Planning and SchedulingSearch and Optimization: Sampling/Simulation-based SearchMultiagent Systems: Multiagent Learning

Application Category

Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystemsGraph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphsResponsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We investigate adaptive minimal routing in 2D torus networks on chip NoCs under node fault conditions comparing a reinforcement learning RL based strategy to an adaptive routing baseline A torus topology is used for its low diameter high connectivity properties The RL approach models each router as an agent that learns to forward packets based on network state while the adaptive scheme uses fixed minimal paths with simple rerouting around faults We implement both methods in simulation injecting up to 50 node faults uniformly at random Key metrics are measured 1 throughput vs offered load at fault density 02 2 packet delivery ratio PDR vs fault density and 3 a fault adaptive score FT vs fault density Experimental results show the RL method achieves significantly higher throughput at high load approximately 2030 gain and maintains higher reliability under increasing faults The RL router delivers more packets per cycle and adapts to faults by exploiting path diversity whereas the adaptive scheme degrades sharply as faults accumulate In particular the RL approach preserves end to end connectivity longer PDR remains above 90 until approximately 3040 faults while adaptive PDR drops to approximately 70 at the same point The fault adaptive score likewise favors RL routing Thus RL based adaptive routing demonstrates clear advantages in throughput and fault resilience for torus NoCs

Problem

• Develops RL-based routing for self-healing 2D torus NoCs under node faults
• Compares RL routing to adaptive baseline on throughput and reliability metrics
• Demonstrates RL's superior fault resilience and throughput in torus networks

Innovation

• Reinforcement learning agents learn packet routing dynamically
• Adaptive routing exploits path diversity to handle node faults
• RL method achieves higher throughput and fault resilience