Curiosity by Design: An LLM-based Coding Assistant Asking Clarification Questions

2025-07-28

AI Summary

Large language model (LLM)-based programming assistants often fail to accurately infer user intent or generate correct code under ambiguous, underspecified prompts. Method: This paper proposes an end-to-end framework with active clarification capability: first, a query classifier detects ambiguous programming requests; then, a fine-tuned LLM generates semantically precise, context-aware clarification questions—mimicking the interactive understanding process in human code review. Contribution/Results: To our knowledge, this is the first work to jointly model the full pipeline from ambiguity detection to clarification generation. It significantly outperforms zero-shot prompting baselines. Both automated evaluation and user studies demonstrate that the generated clarification questions are more informative and practically useful, leading to substantial improvements in downstream code generation accuracy and user satisfaction.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Reasoning under Uncertainty: Probabilistic Programming

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large Language Models (LLMs) are increasingly used as coding assistants. However, the ambiguity of the developer's prompt often leads to incorrect code generation, as current models struggle to infer user intent without extensive prompt engineering or external context. This work aims to build an LLM-based coding assistant that mimics the human code review process by asking clarification questions when faced with ambiguous or under-specified queries. Our end-to-end system includes (1) a query classifier trained to detect unclear programming-related queries and (2) a fine-tuned LLM that generates clarification questions. Our evaluation shows that the fine-tuned LLM outperforms standard zero-shot prompting in generating useful clarification questions. Furthermore, our user study indicates that users find the clarification questions generated by our model to outperform the baseline, demonstrating that our coding assistant produces more accurate and helpful code responses compared to baseline coding assistants.

Problem

• LLMs struggle with ambiguous developer prompts
• Current models lack intent inference without context
• Need for coding assistants that ask clarifying questions

Innovation

• LLM-based coding assistant asks clarification questions
• Query classifier detects unclear programming queries
• Fine-tuned LLM generates useful clarification questions