From App Features to Explanation Needs: Analyzing Correlations and Predictive Potential

2025-08-05

AI Summary

This study investigates whether application metadata—such as version number, rating, and review count—can predict users’ need for behavioral explanations early in the development lifecycle. Leveraging manual annotation and systematic mining of 4,495 metadata-enriched user reviews, we construct the first empirical dataset specifically targeting explainability demand. Using correlation analysis, linear regression, and classification modeling, we find that metadata exhibits only moderate association (r ≈ 0.3–0.4) with security- or system-related explanation needs, while interface-interaction-related needs are virtually unpredictable. Overall, predictive performance is limited. Our key contribution is demonstrating the strong context-dependency of explanation needs, empirically showing that metadata alone is insufficient for robust explainability design—direct user feedback must be integrated. This finding provides both a methodological caution and a practical pathway for early-stage explainability requirement identification.

Technology Category

• Machine Learning: Transparent, Interpretable, Explainable ML
• Computer Vision: Interpretability, Explainability, and Transparency
• Humans and AI: Explainable AI (XAI) for Human Understanding

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

In today's digitized world, software systems must support users in understanding both how to interact with a system and why certain behaviors occur. This study investigates whether explanation needs, classified from user reviews, can be predicted based on app properties, enabling early consideration during development and large-scale requirements mining. We analyzed a gold standard dataset of 4,495 app reviews enriched with metadata (e.g., app version, ratings, age restriction, in-app purchases). Correlation analyses identified mostly weak associations between app properties and explanation needs, with moderate correlations only for specific features such as app version, number of reviews, and star ratings. Linear regression models showed limited predictive power, with no reliable forecasts across configurations. Validation on a manually labeled dataset of 495 reviews confirmed these findings. Categories such as Security & Privacy and System Behavior showed slightly higher predictive potential, while Interaction and User Interface remained most difficult to predict. Overall, our results highlight that explanation needs are highly context-dependent and cannot be precisely inferred from app metadata alone. Developers and requirements engineers should therefore supplement metadata analysis with direct user feedback to effectively design explainable and user-centered software systems.

Problem

• Predicting user explanation needs from app properties
• Analyzing correlations between app features and explanation needs
• Assessing predictive potential of metadata for explanation requirements

Innovation

• Predict explanation needs from app properties
• Analyze correlations in app review metadata
• Use linear regression for limited prediction