PICTS: A Novel Deep Reinforcement Learning Approach for Dynamic P-I Control in Scanning Probe Microscopy

2025-02-11

AI Summary

To address the challenge of non-adaptive proportional-integral (P-I) controller parameter tuning in scanning probe microscopy (SPM) under dynamic imaging conditions, this paper proposes and implements the first deep reinforcement learning framework embedded with a closed-loop control architecture: the Parallel Integrated Control and Training System (PICTS). PICTS introduces a novel end-to-end differentiable mechanism for real-time P-I parameter adaptation, integrating the proximal policy optimization (PPO) algorithm, real-time embedded control, simulation-physical co-training, and hardware-in-the-loop (HIL) interfacing. Experimental validation on atomic force microscopy (AFM) demonstrates a 42% reduction in tracking error, a 2.3× increase in scan speed, and sustained 98.7% closed-loop stability over topographic discontinuities. The framework significantly enhances generalization across diverse operational conditions and enables robust online learning capability.

Technology Category

• Intelligent Robots: Behavior Learning & Control
• Search and Optimization: Learning to Search
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We have developed a Parallel Integrated Control and Training System, leveraging the deep reinforcement learning to dynamically adjust the control strategies in real time for scanning probe microscopy techniques.

Problem

• Dynamic P-I Control
• Deep Reinforcement Learning
• Scanning Probe Microscopy

Innovation

• Deep Reinforcement Learning
• Dynamic Control Adjustment
• Scanning Probe Microscopy