Designing and Evaluating an Educational Recommender System with Different Levels of User Control

2025-01-22

IntRS@RecSys

AI Summary

Current educational recommendation systems suffer from insufficient user control and transparency, hindering effective personalized learning. Method: We propose a novel recommendation system supporting three-tiered (input–process–output) user control—first achieving progressive controllability across user profiling, algorithmic logic, and recommendation outcomes. An interactive prototype was evaluated in an online experiment (N=30), with multi-dimensional user experience assessments and correlation analyses. Results: User control significantly enhanced perceived benefit; transparency moderately correlated with satisfaction (r≈0.5), while trust strongly predicted satisfaction (r≈0.7); transparency and trust showed only weak association. This work establishes both theoretical foundations and practical design patterns for trustworthy educational recommender systems.

Technology Category

• Data Mining & Knowledge Management: Recommender Systems
• Machine Learning: Learning Preferences or Rankings
• Humans and AI: Intelligent User Interfaces

Application Category

• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Responsible Web: Algorithmic accountability and transparency on the web
• Security and Privacy: Data transparency and provenance

Abstract

Educational recommender systems (ERSs) play a crucial role in personalizing learning experiences and enhancing educational outcomes by providing recommendations of personalized resources and activities to learners, tailored to their individual learning needs. However, their effectiveness is often diminished by insufficient user control and limited transparency. To address these challenges, in this paper, we present the systematic design and evaluation of an interactive ERS, in which we introduce different levels of user control. Concretely, we introduce user control around the input (i.e., user profile), process (i.e., recommendation algorithm), and output (i.e., recommendations) of the ERS. To evaluate our system, we conducted an online user study (N=30) to explore the impact of user control on users' perceptions of the ERS in terms of several important user-centric aspects. Moreover, we investigated the effects of user control on multiple recommendation goals, namely transparency, trust, and satisfaction, as well as the interactions between these goals. Our results demonstrate the positive impact of user control on user perceived benefits of the ERS. Moreover, our study shows that user control strongly correlates with transparency and moderately correlates with trust and satisfaction. In terms of interaction between these goals, our results reveal that transparency moderately correlates and trust strongly correlates with satisfaction. Whereas, transparency and trust stand out as less correlated with each other.

Problem

• Educational Recommendation Systems
• Personalized Learning
• Student Control

Innovation

• User-Controlled Recommendation
• Educational Recommendation System
• Transparency and Trust