NiceWebRL: a Python library for human subject experiments with reinforcement learning environments

2025-08-21

AI Summary

This study addresses the bottleneck of adapting JAX-based reinforcement learning (RL) environments for direct use in online human behavioral experiments. To bridge this gap, we introduce NiceWebRL—a novel toolkit enabling seamless, low-latency, high-fidelity deployment of JAX RL environments (e.g., Craftax, Overcooked, XLand-Minigrid) as web-based platforms for human participants. It supports both single- and multi-agent experimental paradigms, integrates WebSocket-based real-time interaction, and provides Flax-compatible interfaces. Our core contributions are threefold: (1) establishing a standardized, cognitively grounded experimental infrastructure to rigorously evaluate and compare computational cognitive RL models against human behavioral data; (2) facilitating generalization assessment of human–AI collaborative algorithms under ecologically valid conditions; and (3) enabling empirical studies of large language model (LLM)-augmented human decision-making. Three cutting-edge applications demonstrate NiceWebRL’s pivotal role as an interdisciplinary bridge between AI and cognitive science.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Application Domains: Humanities & Computational Social Science

Application Category

• Economics, Online Markets and Human Computation: Sustainability of Web economics
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

We present NiceWebRL, a research tool that enables researchers to use machine reinforcement learning (RL) environments for online human subject experiments. NiceWebRL is a Python library that allows any Jax-based environment to be transformed into an online interface, supporting both single-agent and multi-agent environments. As such, NiceWebRL enables AI researchers to compare their algorithms to human performance, cognitive scientists to test ML algorithms as theories for human cognition, and multi-agent researchers to develop algorithms for human-AI collaboration. We showcase NiceWebRL with 3 case studies that demonstrate its potential to help develop Human-like AI, Human-compatible AI, and Human-assistive AI. In the first case study (Human-like AI), NiceWebRL enables the development of a novel RL model of cognition. Here, NiceWebRL facilitates testing this model against human participants in both a grid world and Craftax, a 2D Minecraft domain. In our second case study (Human-compatible AI), NiceWebRL enables the development of a novel multi-agent RL algorithm that can generalize to human partners in the Overcooked domain. Finally, in our third case study (Human-assistive AI), we show how NiceWebRL can allow researchers to study how an LLM can assist humans on complex tasks in XLand-Minigrid, an environment with millions of hierarchical tasks. The library is available at https://github.com/KempnerInstitute/nicewebrl.

Problem

• Develops a Python library for online human experiments with RL environments
• Enables comparison of AI algorithms to human performance in cognition
• Facilitates multi-agent research for human-AI collaboration across domains

Innovation

• Python library for human-RL experiments
• Converts Jax environments to web interface
• Supports single and multi-agent environments