🤖 AI Summary
This study proposes a novel approach leveraging the multimodal large language model Qwen-VL 3.5 to implicitly learn item response theory (IRT) parameterized response curves directly from multiple-choice questions containing both text and images, without requiring explicit parameter fitting. Through supervised fine-tuning and prompt engineering, the model reproduces option-level response probabilities conditioned on student ability, simultaneously modeling both the three-parameter logistic (3PL) model and the multiple-choice model (MCM). Experimental results demonstrate that the method accurately approximates true item difficulty parameters on held-out test sets and effectively captures systematic error patterns across students of varying abilities. To the best of our knowledge, this work represents the first successful end-to-end reconstruction of IRT response functions using a multimodal large language model.
📝 Abstract
We present results from reconstructing multiple-choice model (MCM) and three-parameter logistic (3PL) model curves using a fine-tuned multimodal large language model (LLM) based on Qwen3.5. The model is prompted and fine-tuned to replicate choice probabilities across a large training corpus of multiple-choice items containing both image and text stimuli, conditioned on a labeled set of student ability levels. By learning to reproduce the systematic error patterns of students across a discrete range of abilities, the LLM implicitly captures the underlying response probabilities encoded in the 3PL and MCM curves. This allows us to accurately approximate item difficulty on a held-out test set directly from the model's predicted option probabilities.