Problem
Research questions and friction points this paper is trying to address.
automated essay scoring
corpus generalizability
multi-trait scoring
cross-prompt scoring
annotated corpus
ICLE++: Modeling Fine-Grained Traits for Holistic Essay Scoring
🤖 AI Summary
This study addresses the limited cross-corpus generalization and lack of fine-grained, multi-dimensional scoring in existing automated essay scoring (AES) models, which are typically evaluated on single datasets such as ASAP. To bridge this gap, the authors construct the ICLE++ corpus, providing the first comprehensive human annotations for holistic scores and multiple fine-grained writing traits on argumentative essays from ICLE. Building upon this resource, they introduce a multi-task learning evaluation framework that establishes a new benchmark for assessing AES models’ cross-corpus generalization, multi-trait scoring capability, and prompt transferability. This work fills a critical void in the AES field by delivering a high-quality, multi-dimensional annotated dataset, thereby advancing research toward more realistic and complex educational applications.
📝 Abstract
The majority of the recently-developed models for automated essay scoring (AES) are evaluated solely on the ASAP corpus. However, ASAP is not without its limitations. For instance, it is not clear whether models trained on ASAP can generalize well when evaluated on other corpora. In light of these limitations, we introduce ICLE++, a corpus of persuasive student essays annotated with both holistic scores and trait-specific scores. Not only can ICLE++ be used to test the generalizability of AES models trained on ASAP, but it can also facilitate the evaluation of models developed for newer AES problems such as multi-trait scoring and cross-prompt scoring. We believe that ICLE++, which represents a culmination of our long-term effort in annotating the essays in the ICLE corpus, contributes to the set of much-needed annotated corpora for AES research.
Innovation
Methods, ideas, or system contributions that make the work stand out.
ICLE++
automated essay scoring
trait-specific scoring
cross-prompt scoring
annotated corpus
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
