Responsible Institutional Analytics: Interpreting Bias with AI Support

2026-10-05

AI Summary

This study addresses interpretive biases in higher education analytics dashboards arising from data limitations and contextual omissions by proposing the FACTRIA framework. This framework pioneers the deep integration of structured bias factors with generative AI, employing a chatbot interface to guide users in reflecting on potential biases and thereby enabling context-aware interactive analytical support. Through qualitative research combined with network analysis methods, empirical findings demonstrate that this approach effectively helps users identify overlooked influencing factors and significantly enhances the objectivity and accountability of data interpretation. Ultimately, this work provides an innovative paradigm for responsible educational data analytics.

Abstract

Institutional Analytics (IA) dashboards inform decision-making in higher education, yet data limitations, constraints in analytical techniques, and missing contextual information often affect their interpretation. To support more responsible interpretation of IA, we introduce FACTRIA, a framework that organizes potential biasing factors across four areas: the analytics pipeline, institutional context, course-level characteristics, and demographics. We used the FACTRIA framework as input to a generative-AI chatbot designed to prompt users to reflect on these factors while analyzing IA. A qualitative study with stakeholders, drawing on four authentic IA cases, and a transition network analysis showed that the chatbot prompted participants to recognize how overlooked factors influenced their initial interpretation. Findings indicated that combining a structured framework with AI-based guidance can enhance context-aware, responsible interpretation of institutional data.

Problem

• Institutional Analytics
• Bias Interpretation
• Responsible AI
• Higher Education

Innovation

• Institutional Analytics
• FACTRIA framework
• Generative AI
• Bias interpretation
• Responsible analytics