Problem
Research questions and friction points this paper is trying to address.
cognitive modeling
algorithmic reasoning
behavioral data
program refinement
large language models
Hypothesis-guided discovery of cognitive algorithms via program refinement
🤖 AI Summary
This study addresses the limited flexibility of traditional cognitive models and the lack of human prior knowledge in large language models (LLMs) by proposing a human-machine hybrid framework that reformulates cognitive algorithm discovery as a program refinement task. By integrating probabilistic programming, Bayesian inference, and LLM-based agents, the framework enables LLMs to automatically identify biases and optimize code under human prior constraints, thereby balancing interpretability with scalability. Experimental results demonstrate that the refined models significantly improve goodness-of-fit to behavioral data while uncovering key algorithmic innovations essential for capturing behavioral variability.
📝 Abstract
Developing cognitive models of algorithmic reasoning from behavioral data is a central problem in cognitive science that challenges current methods. Traditional approaches to cognitive modeling are interpretable and benefit from human expertise, but lack flexibility and scalability. Emerging techniques using large language models (LLMs) for de novo generation of cognitive models are scalable and flexible, but lack a role for human expertise and have mostly been applied to simpler tasks than algorithm recovery. We propose a hybrid system that treats discovery of cognitive algorithms as a program refinement problem. Human-created cognitive models are expressed as probabilistic programs and provided to a system of LLM agents with a mandate to: identify mismatches between model and behavior; propose code-level modifications within researcher-specified constraints; and verify structural fidelity. Revisions propagate to a probabilistic inference module that performs inference for latent variables and data likelihood computations. We evaluate the pipeline on human behavior in a problem-solving paradigm that exposes a variety of cognitive algorithms. Revised models consistently improve model fit relative to ancestral models and reveal a small set of recurring innovations that capture meaningful behavioral variability in this task.
Innovation
Methods, ideas, or system contributions that make the work stand out.
cognitive modeling
program refinement
large language models
probabilistic programs
hybrid system
🔎 Similar Papers
No similar papers found.
