🤖 AI Summary
Traditional data augmentation techniques (e.g., rotation, flipping) only perturb low-level geometric attributes of images and cannot control high-level semantics (e.g., animal species, plant categories), resulting in insufficient semantic diversity in few-shot learning scenarios. To address this, we propose the first fine-tuning-free, semantic-level image augmentation framework leveraging frozen pre-trained text-to-image diffusion models (e.g., Stable Diffusion). Our method integrates CLIP-guided latent-space editing, prompt-driven semantic redrawing, and conditional inversion to enable zero-shot cross-species and cross-category semantic editing. Crucially, it requires no additional training and generalizes to unseen concepts. Evaluated on few-shot classification and real-world agricultural weed recognition tasks, our approach improves average accuracy by 4.2–9.7%, demonstrating the critical role of semantic diversity in enhancing downstream task performance.
📝 Abstract
Data augmentation is one of the most prevalent tools in deep learning, underpinning many recent advances, including those from classification, generative models, and representation learning. The standard approach to data augmentation combines simple transformations like rotations and flips to generate new images from existing ones. However, these new images lack diversity along key semantic axes present in the data. Current augmentations cannot alter the high-level semantic attributes, such as animal species present in a scene, to enhance the diversity of data. We address the lack of diversity in data augmentation with image-to-image transformations parameterized by pre-trained text-to-image diffusion models. Our method edits images to change their semantics using an off-the-shelf diffusion model, and generalizes to novel visual concepts from a few labelled examples. We evaluate our approach on few-shot image classification tasks, and on a real-world weed recognition task, and observe an improvement in accuracy in tested domains.