Sophisticated Policies from Epistemic Priors

2026-07-21

AI Summary

This work addresses the challenge of integrating information seeking with closed-loop control in active inference. It proposes a cognitive-prior variational free energy framework that enables closed-loop planning by jointly modeling the posterior over states and actions, explicitly incorporating the dependence of future actions on anticipated states into the inference process. The key insight is that the advantage of complex reasoning stems from the closed-loop architecture itself rather than tree search per se, a mechanism unified under the notion of cognitive priors. Experiments on the Reactivity Maze benchmark demonstrate that neither purely cognition-driven nor open-loop strategies succeed in completing the task, whereas the proposed method achieves robust goal-directed behavior.

Abstract

Sophisticated Inference is a variant of active inference often associated with recursive belief modeling and tree search. We argue that its central computational role is simpler: within a planning horizon, it makes active inference closed-loop by allowing future actions to depend on future states and observations. This closed-loop structure can be represented in the epistemic-prior variational free energy framework. Epistemic priors supply the active-inference objective, while a joint posterior over future states and actions supplies the state-contingent control structure. We evaluate this decomposition in the Reactivity Maze, a stochastic benchmark designed to separate epistemic incentive from inner-horizon closed-loop control. The comparison includes three variational objectives with the same state-action posterior family, an action-state factorized active inference objective, Sophisticated Inference, and standard Expected Free Energy planning. The results show that neither ingredient is sufficient on its own. Methods without an epistemic component do not seek information, while methods that prevent future actions from depending on future states cannot turn information into reliable goal-reaching. By contrast, both Sophisticated Inference and full-joint epistemic-prior active inference solve the environment by combining epistemic drive with closed-loop inference. These results show that the advantage associated with Sophisticated Inference need not be specific to tree search itself. It arises from the closed-loop form of active inference, and this form can be represented in epistemic-prior variational inference when the posterior keeps future actions dependent on future states.

Problem

• active inference
• epistemic priors
• closed-loop control
• sophisticated inference
• goal-directed behavior

Innovation

• Sophisticated Inference
• Epistemic Priors
• Active Inference
• Closed-loop Control
• Variational Free Energy