🤖 AI Summary
Low-light conditions and occlusions severely degrade thermal imaging-based weapon segmentation accuracy. Method: This paper pioneers the systematic application of Vision Transformers (ViTs) to this task, introducing the first large-scale thermal weapon segmentation dataset—comprising 9,711 real-world surveillance images—and conducting a rigorous comparative evaluation of SegFormer, Swin Transformer, SegNeXt, and DeepLabV3+ within the MMSegmentation framework. To enhance data quality and generalization, SAM2-assisted annotation and standardized augmentation strategies are integrated. Results: SegFormer-b5 achieves state-of-the-art performance with 94.15% mIoU and 97.04% pixel accuracy; SegFormer-b0 attains real-time inference at 98.32 FPS. ViT-based architectures consistently outperform CNNs, demonstrating superior capability in modeling long-range dependencies and capturing fine-grained structural details. This work establishes a new high-accuracy, high-efficiency segmentation paradigm for thermal-imaging-based security applications.
📝 Abstract
Thermal weapon segmentation is crucial for surveillance and security applications, enabling robust detection under lowlight and visually obscured conditions where RGB-based systems fail. While convolutional neural networks (CNNs) dominate thermal segmentation literature, their ability to capture long-range dependencies and fine structural details is limited. Vision Transformers (ViTs), with their global context modeling capabilities, have achieved state-of-the-art results in RGB segmentation tasks, yet their potential in thermal weapon segmentation remains underexplored. This work adapts and evaluates four transformer-based architectures SegFormer, DeepLabV3+, SegNeXt, and Swin Transformer for binary weapon segmentation on a custom thermal dataset comprising 9,711 images collected from real world surveillance videos and automatically annotated using SAM2. We employ standard augmentation strategies within the MMSegmentation framework to ensure robust model training and fair architectural comparison. Experimental results demonstrate significant improvements in segmentation performance: SegFormer-b5 achieves the highest mIoU (94.15%) and Pixel Accuracy (97.04%), while SegFormer-b0 provides the fastest inference speed (98.32 FPS) with competitive mIoU (90.84%). SegNeXt-mscans offers balanced performance with 85.12 FPS and 92.24% mIoU, and DeepLabV3+ R101-D8 reaches 92.76% mIoU at 29.86 FPS. The transformer architectures demonstrate robust generalization capabilities for weapon detection in low-light and occluded thermal environments, with flexible accuracy-speed trade-offs suitable for diverse real-time security applications.