Optimal navigation of magnetic artificial microswimmers in blood capillaries with deep reinforcement learning

2024-03-29

AI Summary

Precise navigation of magnetic artificial microswimmers (ABFs) within capillary networks under realistic blood flow conditions remains challenging due to complex hemodynamics and prohibitive computational costs of high-fidelity simulations. Method: We propose a cross-scale control framework integrating off-policy Actor-Critic deep reinforcement learning with a reduced-order blood flow model. Contribution/Results: To our knowledge, this is the first work achieving robust policy transfer from the reduced-order model to high-fidelity, multiscale hemodynamic simulations—including hydrodynamic interactions of red blood cells—in retinal capillaries. The framework enables successful ABF-targeted navigation while drastically reducing computational overhead, ensuring real-time closed-loop magnetic field control. The learned policy converges stably to the designated target in both the reduced-order and full-scale models. This approach demonstrates strong potential for clinical deployment in targeted drug delivery and minimally invasive diagnostics.

Technology Category

• Multiagent Systems: Adversarial Agents
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Behavior Learning & Control

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Biomedical applications such as targeted drug delivery, microsurgery, and sensing rely on reaching precise areas within the body in a minimally invasive way. Artificial bacterial flagella (ABFs) have emerged as potential tools for this task by navigating through the circulatory system with the help of external magnetic fields. While their swimming characteristics are well understood in simple settings, their controlled navigation through realistic capillary networks remains a significant challenge due to the complexity of blood flow and the high computational cost of detailed simulations. We address this challenge by conducting numerical simulations of ABFs in retinal capillaries, propelled by an external magnetic field. The simulations are based on a validated blood model that predicts the dynamics of individual red blood cells and their hydrodynamic interactions with ABFs. The magnetic field follows a control policy that brings the ABF to a prescribed target. The control policy is learned with an actor-critic, off-policy reinforcement learning algorithm coupled with a reduced-order model of the system. We show that the same policy robustly guides the ABF to a prescribed target in both the reduced-order model and the fine-grained blood simulations. This approach is suitable for designing robust control policies for personalized medicine at moderate computational cost.

Problem

• Navigating artificial microswimmers in complex blood capillaries
• Overcoming high computational costs in detailed simulations
• Achieving robust control for targeted biomedical applications

Innovation

• Deep reinforcement learning controls ABF navigation
• Reduced-order model enables efficient policy learning
• Validated blood simulation ensures realistic dynamics