Towards an automatic method for generating topical vocabulary test forms for specific reading passages

2025-05-24

AI Summary

STEM reading materials lack lightweight, ready-to-deploy tools for assessing students’ domain-specific background knowledge. Method: This study proposes K-tool, the first system enabling single-text-driven, corpus-free automatic generation of domain vocabulary tests. It identifies the core domain via topic detection, models lexical semantic relationships using word embeddings and co-occurrence features, and automatically selects highly relevant target words and generates semantically plausible distractors. The system supports real-time diagnostic assessment and knowledge-state prediction for middle and high school students. Contribution/Results: The architecture is empirically validated; preliminary experiments demonstrate that generated tests significantly differentiate students across knowledge levels (p < 0.01), indicating potential for just-in-time instructional intervention. Its core innovation lies in decoupling test generation from pre-built corpora, thereby enabling “one-text-one-test” lightweight, deployable automated assessment.

Technology Category

• Data Mining & Knowledge Management: Knowledge Acquisition from the Web
• Knowledge Representation and Reasoning: Knowledge Acquisition
• Natural Language Processing: Generation

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Background knowledge is typically needed for successful comprehension of topical and domain specific reading passages, such as in the STEM domain. However, there are few automated measures of student knowledge that can be readily deployed and scored in time to make predictions on whether a given student will likely be able to understand a specific content area text. In this paper, we present our effort in developing K-tool, an automated system for generating topical vocabulary tests that measure students' background knowledge related to a specific text. The system automatically detects the topic of a given text and produces topical vocabulary items based on their relationship with the topic. This information is used to automatically generate background knowledge forms that contain words that are highly related to the topic and words that share similar features but do not share high associations to the topic. Prior research indicates that performance on such tasks can help determine whether a student is likely to understand a particular text based on their knowledge state. The described system is intended for use with middle and high school student population of native speakers of English. It is designed to handle single reading passages and is not dependent on any corpus or text collection. In this paper, we describe the system architecture and present an initial evaluation of the system outputs.

Problem

• Automated measurement of student background knowledge for text comprehension
• Generating topical vocabulary tests for specific reading passages
• Assessing student readiness to understand STEM domain texts

Innovation

• Automated system generates topical vocabulary tests
• Detects text topics and produces related vocabulary items
• Generates knowledge forms with topic-related and unrelated words