A Reinforcement Learning-Based Telematic Routing Protocol for the Internet of Underwater Things

2025-05-30

AI Summary

Conventional RPL is ill-suited for the Underwater Internet of Things (IoUT), which suffers from severe bandwidth constraints, high propagation latency, dynamic node mobility, and stringent energy limitations. To address these challenges, this paper proposes RL-RPL-UA—a lightweight, RPL-compliant reinforcement learning routing protocol. RL-RPL-UA innovatively embeds a resource-efficient Q-learning agent within each RPL node, enabling autonomous parent selection based solely on local state observations: packet delivery ratio (PDR), buffer occupancy, link quality indicator (LQI), and residual energy. It preserves RPL’s standard message formats and control-plane semantics while introducing a dynamically tunable objective function for real-time environmental adaptation. Evaluated in Aqua-Sim, RL-RPL-UA achieves a 9.2% improvement in packet delivery ratio, a 14.8% reduction in per-packet energy consumption, and extends network lifetime by 80 seconds compared to native RPL. This work establishes a deployable, adaptive intelligent routing paradigm tailored for resource-constrained underwater networks.

Technology Category

• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Intelligent Robots: Learning & Optimization for ROB
• Machine Learning: Reinforcement Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The Internet of Underwater Things (IoUT) faces major challenges such as low bandwidth, high latency, mobility, and limited energy resources. Traditional routing protocols like RPL, which were designed for land-based networks, do not perform well in these underwater conditions. This paper introduces RL-RPL-UA, a new routing protocol that uses reinforcement learning to improve performance in underwater environments. Each node includes a lightweight RL agent that selects the best parent node based on local information such as packet delivery ratio, buffer level, link quality, and remaining energy. RL-RPL-UA keeps full compatibility with standard RPL messages and adds a dynamic objective function to support real-time decision-making. Simulations using Aqua-Sim show that RL-RPL-UA increases packet delivery by up to 9.2%, reduces energy use per packet by 14.8%, and extends network lifetime by 80 seconds compared to traditional methods. These results suggest that RL-RPL-UA is a promising and energy-efficient routing solution for underwater networks.

Problem

• Improving routing performance in underwater IoUT networks
• Enhancing energy efficiency and packet delivery in underwater conditions
• Developing RL-based protocol compatible with standard RPL

Innovation

• Uses reinforcement learning for underwater routing
• Lightweight RL agent optimizes parent node selection
• Dynamic objective function enhances real-time decisions