Merlin's Whisper: Enabling Efficient Reasoning in LLMs via Black-box Adversarial Prompting

2025-10-12

AI Summary

Large reasoning models (LRMs) exhibit strong reasoning capabilities but suffer from excessive computational overhead and latency due to lengthy chain-of-thought (CoT) generation—a phenomenon termed “overthinking.” This work proposes AdvPrompt, the first black-box adversarial prompting framework specifically designed to mitigate overthinking in LRMs. AdvPrompt compresses reasoning paths via multi-perspective input perturbation and iterative prompt refinement, without requiring model access, gradient information, or architectural modifications—ensuring compatibility with both open- and closed-weight models across scales and architectures. Experiments on GSM8K and MATH-500 demonstrate that AdvPrompt reduces average token consumption by ~40% while preserving accuracy. Notably, Qwen3’s response length on GSM8K easy questions is reduced by 3×; Claude-3.7 and Gemini-2.5 achieve 35% and 47% token savings, respectively, on MATH-500.

Technology Category

• Natural Language Processing: Prompt Engineering / Prompting
• Machine Learning: Large Multimodal Models (LMMs)
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large reasoning models (LRMs) have demonstrated remarkable proficiency in tackling complex reasoning tasks through step-by-step thinking. However, such a lengthy reasoning process incurs substantial computational and latency overheads, hindering the practical deployment of these models. In this work, we present a new perspective on mitigating overthinking in LRMs via black-box adversarial prompting. By treating both open-source LRMs and closed-source APIs as black-box communicators, we investigate how to elicit concise responses without sacrificing accuracy. We introduce AdvPrompt, an iterative refinement framework that generates high-quality adversarial prompts from diverse perspectives. Experiments across multiple benchmarks demonstrate that AdvPrompt consistently reduces token usage while preserving performance. Notably, AdvPrompt achieves a 3x reduction in average response length on simple GSM8K questions for the Qwen3 model series, and delivers an average ~40% token reduction across four benchmarks. For closed-source APIs, AdvPrompt reduces token usage on MATH-500 by 35% for Claude-3.7 and 47% for Gemini-2.5. Further analysis reveals the generalizability of AdvPrompt across various model scales and families, underscoring the potential of black-box prompting as a practical and effective strategy for enhancing LRM efficiency.

Problem

• Reducing computational overhead in large reasoning models
• Minimizing token usage while preserving model accuracy
• Addressing overthinking in black-box LLMs via adversarial prompting

Innovation

• Black-box adversarial prompting reduces reasoning overhead
• Iterative refinement framework generates efficient adversarial prompts
• Token usage reduction while preserving model performance accuracy