JU-NLP at Touché: Covert Advertisement in Conversational AI-Generation and Detection Strategies

2025-09-12

AI Summary

This study addresses the transparency and trustworthiness challenges posed by covert advertising—such as soft product placements and intent obfuscation—in conversational AI systems. Methodologically, it proposes a co-designed generation-and-detection framework: (1) a context-aware ad generation model that integrates user query intent and dialogue history, leveraging LLM fine-tuning and advanced prompt engineering for natural, semantically coherent insertion; and (2) a lightweight, response-only detector based on a fine-tuned CrossEncoder with DeBERTa-v3-base, augmented by prompt-based reconstruction to achieve high-accuracy identification without auxiliary inputs. The framework achieves, for the first time, simultaneous breakthroughs in generation stealth—achieving precision of 1.0 and recall of 0.71—and detection robustness—attaining F1 scores of 0.99–1.00 across diverse adversarial settings. This work establishes a novel paradigm for interpretable, auditable, and commercially viable conversational systems.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Search and Retrieval-Augmented AI: Ad search and search for Web retail
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

This paper proposes a comprehensive framework for the generation of covert advertisements within Conversational AI systems, along with robust techniques for their detection. It explores how subtle promotional content can be crafted within AI-generated responses and introduces methods to identify and mitigate such covert advertising strategies. For generation (Sub-Task~1), we propose a novel framework that leverages user context and query intent to produce contextually relevant advertisements. We employ advanced prompting strategies and curate paired training data to fine-tune a large language model (LLM) for enhanced stealthiness. For detection (Sub-Task~2), we explore two effective strategies: a fine-tuned CrossEncoder ( exttt{all-mpnet-base-v2}) for direct classification, and a prompt-based reformulation using a fine-tuned exttt{DeBERTa-v3-base} model. Both approaches rely solely on the response text, ensuring practicality for real-world deployment. Experimental results show high effectiveness in both tasks, achieving a precision of 1.0 and recall of 0.71 for ad generation, and F1-scores ranging from 0.99 to 1.00 for ad detection. These results underscore the potential of our methods to balance persuasive communication with transparency in conversational AI.

Problem

• Generating covert advertisements in conversational AI systems
• Detecting subtle promotional content in AI responses
• Balancing persuasive communication with transparency

Innovation

• Fine-tuned LLM for stealthy ad generation
• CrossEncoder for direct ad classification
• Prompt-based DeBERTa for response reformulation