Technology Category
Machine Learning: Large Multimodal Models (LMMs)Computer Vision: Multi-modal VisionNatural Language Processing: Sentence-level Semantics, Textual Inference, etc.
Teaching Sarcasm: Few-Shot Multimodal Sarcasm Detection via Distillation to a Parameter-Efficient Student
🤖 AI Summary
In low-resource multimodal sarcasm detection, modeling image-text incongruence remains challenging, and limited supervision in few-shot settings constrains the effectiveness of parameter-efficient fine-tuning (PEFT) methods. To address these issues, this paper proposes PEKD, an entropy-aware knowledge distillation framework tailored for PEFT. Its core contributions are twofold: (1) an entropy-aware gating mechanism that dynamically suppresses high-uncertainty distillation signals from the teacher model, thereby improving distillation reliability; and (2) a modular distillation architecture that seamlessly integrates with diverse PEFT student models—including LoRA, Adapters, and Prompt Tuning—without architectural modification. Extensive few-shot experiments on two benchmark multimodal datasets demonstrate that PEKD consistently outperforms existing PEFT approaches and large multimodal baselines, achieving state-of-the-art performance.
Technology Category
Application Category
📝 Abstract
Multimodal sarcasm detection is challenging, especially in low-resource settings where subtle image-text contradictions are hard to learn due to scarce annotated data, which hinders the model's performance. Parameter-efficient fine-tuning (PEFT) methods like adapters, LoRA, and prompt tuning reduce overfitting but struggle to reach optimal performance due to limited supervision from few-shot data. We propose PEKD, a unified framework that enhances PEFT methods via distillation from an expert model trained on large-scale sarcasm data, which acts as the teacher. To mitigate unreliable signals from the teacher, we introduce an entropy-aware gating mechanism that dynamically adjusts the distillation strength based on teacher confidence. Experiments on two public datasets demonstrate that our PEKD framework enables PEFT methods to outperform both prior parameter-efficient approaches and large multimodal models, achieving strong results in the few-shot scenario. The framework is modular and adaptable to a wide range of multimodal models and tasks.
Problem
Research questions and friction points this paper is trying to address.
Detecting sarcasm from limited multimodal data
Improving parameter-efficient methods with few-shot supervision
Mitigating unreliable signals in multimodal sarcasm detection
Innovation
Methods, ideas, or system contributions that make the work stand out.
Distillation from expert model enhances PEFT methods
Entropy-aware gating adjusts distillation strength dynamically
Modular framework adapts to multimodal models and tasks
🔎 Similar Papers
No similar papers found.
