Intrinsically-Motivated Humans and Agents in Open-World Exploration
🤖 AI Summary
This study investigates how intrinsic motivation drives divergent exploration behaviors between humans (adults and children) and AI agents in the open-world environment Crafter.
Method: We conducted behavioral experiments, collected private speech annotations, and computationally modeled three intrinsic motivation signals—entropy, information gain, and empowerment—to systematically quantify and compare cross-subject exploration dynamics.
Contribution/Results: We find, for the first time, that entropy and empowerment consistently correlate positively with human exploration progress: entropy primarily governs early-stage state diversity, while empowerment underpins later-stage control acquisition; information gain shows weaker associations. Notably, children’s goal-directed verbalizations significantly enhance exploration efficiency. Critically, the entropy–empowerment synergy better captures human exploration logic than either metric alone, providing empirical grounding and a computationally tractable pathway for designing human-like exploration strategies in embodied AI agents.
Application Category
📝 Abstract
What drives exploration? Understanding intrinsic motivation is a long-standing challenge in both cognitive science and artificial intelligence; numerous objectives have been proposed and used to train agents, yet there remains a gap between human and agent exploration. We directly compare adults, children, and AI agents in a complex open-ended environment, Crafter, and study how common intrinsic objectives: Entropy, Information Gain, and Empowerment, relate to their behavior. We find that only Entropy and Empowerment are consistently positively correlated with human exploration progress, indicating that these objectives may better inform intrinsic reward design for agents. Furthermore, across agents and humans we observe that Entropy initially increases rapidly, then plateaus, while Empowerment increases continuously, suggesting that state diversity may provide more signal in early exploration, while advanced exploration should prioritize control. Finally, we find preliminary evidence that private speech utterances, and particularly goal verbalizations, may aid exploration in children.
Problem
Research questions and friction points this paper is trying to address.
Compare human and AI exploration in open-world environments
Identify intrinsic motivations (Entropy, Empowerment) linked to human exploration
Study role of verbalizations in children's exploration strategies
Innovation
Methods, ideas, or system contributions that make the work stand out.
Compare humans and AI in open-world exploration
Use Entropy and Empowerment as intrinsic rewards
Study verbalizations aiding child exploration
🔎 Similar Papers
No similar papers found.
