MOAT: Model-Agnostic Randomized Transformations for preventing Efficiency Degradation Attacks on ViTs

2026-08-05

AI Summary

This work addresses the vulnerability of Vision Transformers (ViTs) to adversarial attacks that exploit token-pruning mechanisms in resource-constrained settings, often causing severe computational overhead. To counter this, the authors propose MOAT—a model-agnostic input preprocessing defense framework that leverages randomized input transformations to effectively mitigate efficiency-targeted adversarial attacks without altering the model architecture or pruning strategy. Designed as a plug-and-play solution, MOAT is compatible with diverse ViT variants and token-pruning methods. Experimental results demonstrate that MOAT consistently limits the degradation in computational cost induced by adversarial attacks to within 3.4% of the original, unattacked GFLOPs across all evaluated models, thereby preserving computational efficiency under attack.

Abstract

To adopt the Vision Transformers (ViTs) in resource-constrained environment, token pruning is widely used to reduce computational cost without impacting accuracy. However, adversaries have developed targeted attacks against said token pruning techniques to undermine such attempts to make ViTs efficient. In this paper, we propose MOAT, a model-agnostic pre-processing defense pipeline that applies a combination of input transformations to protect efficient ViT implementations against adversarial efficiency attacks. MOAT operates directly on the input without requiring modifications to the model architecture or token pruning mechanism. Experimental results demonstrate that, across all evaluated ViT models, MOAT limits GFLOPs degradation under adversarial attacks to within 3.4% of the original unattacked model.

Problem

• Vision Transformers
• token pruning
• efficiency degradation attacks
• adversarial attacks
• model efficiency

Innovation

• Model-Agnostic
• Randomized Transformations
• Efficiency Degradation Attacks
• Vision Transformers
• Token Pruning