Digital Twin-Empowered Deep Reinforcement Learning for Intelligent VNF Migration in Edge-Core Networks

2025-08-28

AI Summary

To address the joint optimization challenge of low latency and energy efficiency in Virtual Network Function (VNF) migration within edge-core networks, this paper proposes an intelligent co-optimization framework integrating digital twin (DT) technology with deep reinforcement learning (DRL). Methodologically: (1) a multi-task variational autoencoder–LSTM hybrid DT module is designed to faithfully model environmental dynamics and generate high-fidelity synthetic experiences; (2) VNF migration is formulated as a Markov decision process, and an advantage actor–critic (A2C) algorithm enables adaptive, real-time migration decisions. The key innovation lies in the first integration of a DT-driven experience augmentation mechanism into DRL-based VNF migration, significantly accelerating policy convergence and enhancing generalization. Experimental results demonstrate that the proposed framework reduces average end-to-end latency and energy consumption by 28.6% and 34.1%, respectively, compared to state-of-the-art approaches, establishing a new benchmark for intelligent VNF migration.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

The growing demand for services and the rapid deployment of virtualized network functions (VNFs) pose significant challenges for achieving low-latency and energy-efficient orchestration in modern edge-core network infrastructures. To address these challenges, this study proposes a Digital Twin (DT)-empowered Deep Reinforcement Learning framework for intelligent VNF migration that jointly minimizes average end-to-end (E2E) delay and energy consumption. By formulating the VNF migration problem as a Markov Decision Process and utilizing the Advantage Actor-Critic model, the proposed framework enables adaptive and real-time migration decisions. A key innovation of the proposed framework is the integration of a DT module composed of a multi-task Variational Autoencoder and a multi-task Long Short-Term Memory network. This combination collectively simulates environment dynamics and generates high-quality synthetic experiences, significantly enhancing training efficiency and accelerating policy convergence. Simulation results demonstrate substantial performance gains, such as significant reductions in both average E2E delay and energy consumption, thereby establishing new benchmarks for intelligent VNF migration in edge-core networks.

Problem

• Minimizing latency and energy in VNF orchestration
• Enabling adaptive real-time VNF migration decisions
• Enhancing training efficiency for network optimization

Innovation

• Digital Twin-empowered Deep Reinforcement Learning framework
• Markov Decision Process with Advantage Actor-Critic model
• Multi-task Variational Autoencoder and LSTM network integration