🤖 AI Summary
This study addresses the scarcity of high-quality, comprehensive, and secure multiple-choice questions in natural science education, which are costly to develop manually. The authors propose a human-AI collaborative two-stage generation framework: first, question prototypes are constructed using cognitive modeling templates; then, example-driven, single-step model expansion produces diverse item families targeting the same learning objective. The approach innovatively integrates multi-agent AI prompting with iterative human review to ensure generated items are not mere paraphrases and meet rigorous quality standards. Experimental results show that approximately 50% of the generated items are usable as-is, and minor revisions substantially increase the acceptance rate of entire item families while significantly improving prototype quality.
📝 Abstract
Multiple-choice questions (MCQs) are widely used across diverse educational fields and levels. Well-designed MCQs should evaluate knowledge application in real-world situations. However, writing such test items in sufficient numbers is challenging and time-consuming, especially in natural science education. The problem of a sufficient number of MCQs has two aspects: content coverage and exam security. Therefore, generating test items involves two tasks: creating MCQ prototypes and transforming these prototypes into item series. In automated item generation, prototype creation aligns with template-based methods like cognitive modelling, while item expansion corresponds to example-based techniques. The aim of this research was designing the goal-oriented conceptual model of human - AI co-creation of MCQs that should meet strictly formulated quality criteria. The resulting three-step model for creating MCQ prototypes distributed prompts between several AIs, with human revision of responses for each prompt before setting the next one. To transform the MCQ prototype into an MCQ series, a one-step model was developed in which multiple new items are generated simultaneously. These items assess the same learning outcome but are not simple rephrasings of the prototype or of one another. Based on human and automated evaluation, approximately half of the output MCQs were acceptable without editing. Minor corrections of initially rejected test items allowed for a moderate increase in acceptance of MCQs in series and a significant improvement of MCQ-prototypes.