Adversarial construction as a potential solution to the experiment design problem in large task spaces

2026-02-03

AI Summary

This work addresses the challenge of efficiently exploring universal models of human behavior in high-dimensional task spaces, where conventional random sampling proves inadequate. Focusing on binary sequence prediction tasks, the authors propose an adversarial construction strategy that leverages a hidden Markov model (HMM) to represent the task space and actively generates task instances most likely to elicit novel behavioral patterns. By concentrating experimental design on regions critical for behavioral diversity, this approach substantially outperforms random sampling, uncovering a greater number of qualitatively new phenomena with fewer experiments. The method thus establishes an efficient and practical paradigm for constructing generalizable models of human behavior.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Despite decades of work, we still lack a robust, task-general theory of human behavior even in the simplest domains. In this paper we tackle the generality problem head-on, by aiming to develop a unified model for all tasks embedded in a task-space. In particular we consider the space of binary sequence prediction tasks where the observations are generated by the space parameterized by hidden Markov models (HMM). As the space of tasks is large, experimental exploration of the entire space is infeasible. To solve this problem we propose the adversarial construction approach, which helps identify tasks that are most likely to elicit a qualitatively novel behavior. Our results suggest that adversarial construction significantly outperforms random sampling of environments and therefore could be used as a proxy for optimal experimental design in high-dimensional task spaces.

Problem

• experimental design
• task space
• human behavior
• binary sequence prediction
• high-dimensional

Innovation

• adversarial construction
• experimental design
• task space
• hidden Markov models
• binary sequence prediction