Anchoring Adversarial Trajectories to Data Manifolds: A Bilevel Transfer Optimization Framework

2026-09-30

AI Summary

This study addresses the failure of adversarial transferability caused by surrogate model overfitting due to trajectory geometric disconnection. To this end, we propose a manifold-anchored bilevel transfer framework. Specifically, the method introduces a relaxed manifold anchoring operator as a semantic corrector to constrain adversarial trajectories within a shared semantic subspace. Furthermore, it constructs a geometry-aligned distributional bilevel optimization model, efficiently learning initializations via a Hessian-free linear-time solver to suppress off-manifold noise. Extensive evaluations across ten baselines, twenty-eight configurations, and multiple defense mechanisms demonstrate that the proposed framework significantly enhances adversarial transferability.

Abstract

A key bottleneck in adversarial transfer is a trajectory-level geometric disconnect: ambient gradients often drift away from the intrinsic data manifold, causing surrogate-specific overfitting. To rectify this, we propose Manifold Anchored Bilevel Transfer (MABT), a unified framework that anchors adversarial trajectories to the shared semantic subspace. MABT introduces a relaxed manifold-anchoring operator as a semantic rectifier to suppress off-manifold noise. With this constraint, we cast transfer attack generation as a distributional bilevel optimization problem that learns a geometry-aligned initialization by minimizing expected transfer risk under a surrogate uncertainty distribution. We further develop a Hessian-free solver with linear-time complexity to handle the resulting hierarchy. Experiments demonstrate improved transferability for 10 baseline attackers across 28 attack configurations, diverse victim architectures, and defense mechanisms.

Problem

• adversarial transferability
• data manifold
• trajectory-level geometric disconnect
• surrogate overfitting
• transfer attack

Innovation

• Manifold Anchoring
• Bilevel Optimization
• Adversarial Transferability
• Hessian-free Solver
• Distributional Optimization