Representing Visual Evidence for Item Difficulty Prediction: Visual Textualization and Image-Native Modeling

2026-08-05

AI Summary

This study addresses the challenge of predicting the difficulty of novel mathematics problems in the absence of student response data by systematically evaluating three input representations: plain text, visual-to-textual descriptions (converting images into language), and native image modeling. Leveraging large language models (LLMs) and vision-language models (VLMs) for difficulty regression, the work reveals—for the first time—the complementary yet distinct error patterns and computational pathways between visual-to-textual and native image-based approaches. Experimental results demonstrate that both visual methods consistently outperform plain text. Specifically, Open-VLM–generated textual descriptions uniformly reduce RMSE across diverse LLMs, while end-to-end fine-tuned native image VLMs achieve optimal performance within their own architectural framework.

Abstract

Predicting item difficulty from content can provide an initial estimate for newly developed questions before sufficient student responses are available. Existing approaches typically represent the question stem and answer choices as text. When mathematics items contain visual components, a common pipeline first textualizes that evidence and then applies a text predictor. We ask: how should visual evidence be represented for item difficulty prediction? We compare question text alone, visual textualization, which expresses visual evidence in language, and image-native modeling, which retains the original image. Using Eedi items with difficulty calibrated from student responses, we train large language models (LLMs) and vision-language models (VLMs) directly for difficulty regression. Both visual interfaces achieve the lowest point estimates, although the leading systems cannot be reliably ordered. Open-VLM textualization yields lower RMSE point estimates for all evaluated LLMs, while broader adaptation does so for all image-native VLMs. Test-time interventions show dependence on the paired full-item image, but do not isolate the additional visual component. The two visual interfaces also make partially complementary item-level errors and differ substantially in computational workflow. Thus, textualization should not be treated as the only practical interface: image-native modeling is a competitive alternative whose effectiveness depends on how the VLM is adapted.

Problem

• item difficulty prediction
• visual evidence
• visual textualization
• image-native modeling
• vision-language models

Innovation

• visual textualization
• image-native modeling
• item difficulty prediction
• vision-language models
• large language models