PADFormer: Pose-agnostic Anomaly Detection from Sparse View Images

2026-08-04

AI Summary

This work addresses the reliance on costly 3D reconstruction and multi-view data in pose-invariant anomaly detection by proposing an efficient image-space approach. It introduces cross-view masked reconstruction to this task for the first time, training a Vision Transformer exclusively on normal samples. Through dynamic patch selection, a spatial alignment mechanism, and a multi-mask ensemble strategy, the method directly reconstructs anomaly-free images from sparse reference views without requiring 3D reconstruction. This enables effective handling of large pose variations while achieving state-of-the-art performance on the PAD benchmark. Furthermore, the approach demonstrates strong generalization capability and computational efficiency in few-shot anomaly detection scenarios.

Abstract

Pose-agnostic Anomaly Detection (PAD) remains challenging as anomalies can appear under arbitrary viewpoints, requiring methods to handle significant pose variations. Existing approaches rely on complex 3D reconstruction, which are computationally expensive and require extensive multi-view data. We propose PADFormer, a novel image-space approach that leverages Vision Transformer (ViT) to directly reconstruct anomaly-free versions of query images while preserving pose information. Our key insight is to adapt cross-view masked reconstruction for anomaly detection through training exclusively on normal data, combined with dynamic patch selection and spatial alignment mechanisms that enable effective learning from sparse reference views under significant pose variations. During inference, we perform multiple forward passes with different masking patterns to generate an ensemble of anomaly-free reconstructions, ensuring comprehensive coverage of the query image. Anomalies are detected by comparing these reconstructions with the query image. PADFormer achieves state-of-the-art results on the PAD benchmark while maintaining comparable performance on classic few-shot anomaly detection (FSAD) tasks, demonstrating superior efficiency and generalization without requiring 3D reconstruction.

Problem

• Pose-agnostic Anomaly Detection
• Sparse View Images
• Anomaly Detection
• Pose Variations
• 3D Reconstruction

Innovation

• Pose-agnostic Anomaly Detection
• Vision Transformer
• Masked Image Reconstruction
• Sparse View Learning
• Dynamic Patch Selection