Multi-Scale Visual Prompting for Lightweight Small-Image Classification

2025-12-03

AI Summary

This work addresses the longstanding limitation of visual prompting methods—namely, their exclusive application to large-scale models and high-resolution images—by systematically adapting them to small-image benchmarks (MNIST, Fashion-MNIST, CIFAR-10) for the first time. We propose a multi-scale visual prompting framework that injects learnable global, mesoscopic, and local prompt maps directly into the input space; these are fused via 1×1 convolution and seamlessly integrate with both CNN and ViT backbones. Introducing only 0.02% additional parameters, our method yields significant accuracy improvements across all three datasets. Ablation studies and Grad-CAM visualizations validate both the efficacy and interpretability of the multi-scale design. With negligible computational overhead, our approach establishes the first lightweight, general-purpose, plug-and-play visual prompting solution tailored specifically for small-image classification.

Technology Category

• Computer Vision: Large Vision Models
• Natural Language Processing: Prompt Engineering / Prompting
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Visual prompting has recently emerged as an efficient strategy to adapt vision models using lightweight, learnable parameters injected into the input space. However, prior work mainly targets large Vision Transformers and high-resolution datasets such as ImageNet. In contrast, small-image benchmarks like MNIST, Fashion-MNIST, and CIFAR-10 remain widely used in education, prototyping, and research, yet have received little attention in the context of prompting. In this paper, we introduce extbf{Multi-Scale Visual Prompting (MSVP)}, a simple and generic module that learns a set of global, mid-scale, and local prompt maps fused with the input image via a lightweight $1 imes 1$ convolution. MSVP is backbone-agnostic, adds less than $0.02%$ parameters, and significantly improves performance across CNN and Vision Transformer backbones. We provide a unified benchmark on MNIST, Fashion-MNIST, and CIFAR-10 using a simple CNN, ResNet-18, and a small Vision Transformer. Our method yields consistent improvements with negligible computational overhead. We further include ablations on prompt scales, fusion strategies, and backbone architectures, along with qualitative analyzes using prompt visualizations and Grad-CAM. Our results demonstrate that multi-scale prompting provides an effective inductive bias even on low-resolution images.

Problem

• Adapts vision models for small-image datasets efficiently
• Introduces multi-scale prompting for lightweight classification backbones
• Improves performance on low-resolution benchmarks with minimal parameters

Innovation

• Multi-scale prompting for small-image classification
• Lightweight convolution fusion with global, mid, local prompts
• Backbone-agnostic module adding minimal parameters for performance boost