Model selection for behavioral learning data and applications to contextual bandits

2025-02-18

AI Summary

This study addresses the model selection challenge in modeling individual learning behavior data—such as action sequences—where nonstationarity and observation dependence impede conventional approaches. We propose a novel method for inverting adaptive learning strategies. Our core contribution is a tailored hold-out procedure for nonstationary, dependent data, coupled with a generalized Akaike Information Criterion (AIC) whose theoretical error bound asymptotically approaches the optimal rate under i.i.d. assumptions. The method integrates behavioral modeling, nonstationary time series analysis, and contextual multi-armed bandit frameworks. Empirical validation on synthetic data and human category-learning experiments demonstrates substantial improvements in both explanatory power and predictive accuracy for individual learning dynamics. The approach establishes a new, interpretable, and generalizable paradigm for model selection in cognitive modeling and personalized learning analytics.

Technology Category

• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Machine Learning: Online Learning & Bandits
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Learning for animals or humans is the process that leads to behaviors better adapted to the environment. This process highly depends on the individual that learns and is usually observed only through the individual's actions. This article presents ways to use this individual behavioral data to find the model that best explains how the individual learns. We propose two model selection methods: a general hold-out procedure and an AIC-type criterion, both adapted to non-stationary dependent data. We provide theoretical error bounds for these methods that are close to those of the standard i.i.d. case. To compare these approaches, we apply them to contextual bandit models and illustrate their use on both synthetic and experimental learning data in a human categorization task.

Problem

• Model selection for behavioral data
• Adaptation to non-stationary data
• Application in contextual bandit models

Innovation

• Model selection methods
• Non-stationary dependent data
• Contextual bandit models