🤖 AI Summary
Continuously estimating the dynamic changes in human cognitive capacity remains challenging. This work proposes a theory-driven multimodal learning framework that models cognitive capacity as a two-dimensional physiological state space defined by mental effort and stress. A dual-stream neural network encodes heart rate variability (HRV) and electrodermal activity (EDA) signals separately, which are then integrated via a late fusion strategy coupled with task-specific probabilistic output heads to jointly predict both dimensions. By grounding the two-dimensional physiological representation in established cognitive theories, the approach effectively discriminates between states such as efficient engagement and overload-induced stress. Evaluated on the SWELL-KW dataset, the model achieves balanced accuracies of 70.0% for stress and 72.2% for effort, demonstrating the efficacy of theory-guided supervision and multimodal fusion while sensitively capturing task-induced dynamics in cognitive demand.
📝 Abstract
Human cognitive performance is constrained by limited mental resources, yet continuous computational estimation of cognitive capacity dynamics remains an open challenge. We propose a theory-driven multimodal learning framework that models capacity-related cognitive state as a two-dimensional physiological representation defined by voluntary resource allocation (mental effort) and overload-related strain (stress). The proposed architecture combines dual-stream encoding of cardiac (IBI/HRV) and electrodermal (EDA) signals with late fusion and task-specific output heads that independently estimate probabilistic effort and stress states.
Evaluation on the SWELL-KW dataset using strict leave-one-subject-out cross-validation demonstrates cross-individual generalization (stress: 70.0\% balanced accuracy; effort: 72.2\%), with significant gains from multimodal integration and theory-guided supervision. Rather than collapsing physiological dynamics into a single workload label, the proposed effort--stress state-space enables structured differentiation between distinct cognitive regimes, including productive engagement and overload-related strain. Predicted state trajectories exhibit significant demand-sensitive shifts under controlled workload manipulations, with effort and stress responding differentially across interruption and time-pressure conditions.
These results suggest that physiologically grounded multidimensional state representations may provide a foundation for adaptive systems capable of continuous capacity-aware monitoring and human-centered interaction.