"There Is No Such Thing as a Dumb Question,"But There Are Good Ones

2025-05-15

AI Summary

Prior work lacks a systematic framework for assessing question quality. Method: This paper introduces the first dual-dimensional definition—question appropriateness (capturing sociolinguistic competence in context) and effectiveness (reflecting goal-directed strategic competence)—and builds a dynamically adaptive evaluation framework upon it. We design a semi-adaptive rubric-based scoring system integrating linguistic theory and dynamic contextual modeling, empirically validated on the CAUS and SQUARE datasets. Contribution/Results: The framework significantly improves discriminability between well-formed and defective questions, demonstrating robustness, interpretability, and contextual flexibility across diverse scenarios. Key contributions include: (1) establishing the first theoretically grounded definition of question quality; (2) proposing a scalable, principled assessment paradigm; and (3) providing a reusable foundational tool for evaluating AI question-asking capabilities.

Technology Category

• Natural Language Processing: Question Answering
• Search and Optimization: Evaluation and Analysis
• Knowledge Representation and Reasoning: Qualitative Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Community question answering
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Questioning has become increasingly crucial for both humans and artificial intelligence, yet there remains limited research comprehensively assessing question quality. In response, this study defines good questions and presents a systematic evaluation framework. We propose two key evaluation dimensions: appropriateness (sociolinguistic competence in context) and effectiveness (strategic competence in goal achievement). Based on these foundational dimensions, a rubric-based scoring system was developed. By incorporating dynamic contextual variables, our evaluation framework achieves structure and flexibility through semi-adaptive criteria. The methodology was validated using the CAUS and SQUARE datasets, demonstrating the ability of the framework to access both well-formed and problematic questions while adapting to varied contexts. As we establish a flexible and comprehensive framework for question evaluation, this study takes a significant step toward integrating questioning behavior with structured analytical methods grounded in the intrinsic nature of questioning.

Problem

• Defining and evaluating quality of questions systematically
• Assessing questions via appropriateness and effectiveness dimensions
• Developing adaptable framework for diverse question contexts

Innovation

• Defines good questions with systematic evaluation framework
• Proposes appropriateness and effectiveness as key dimensions
• Develops semi-adaptive rubric-based scoring system