Implementation of reinforcement learning in chemical reaction networks: application to phototaxis as curiosity-driven exploration

2026-06-24

AI Summary

This study addresses a key limitation in traditional phototaxis models, which overlook how single-celled organisms actively reduce perceptual ambiguity through exploration. The authors integrate partially observable Markov decision processes (POMDPs) with chemical reaction networks (CRNs), embedding Bayesian belief updates into CRN ordinary differential equations to jointly optimize light-seeking and information acquisition. For the first time, curiosity-driven reinforcement learning is implemented within a CRN framework, revealing the computational role of tumbling as an information-sampling strategy and establishing a chemically realizable polynomial bound on information gain. By combining a biophysical model of photoreception, inverse reinforcement learning, and the stochastic simulation algorithm (SSA), the approach successfully reproduces the phototactic trajectory distribution of *Chlamydomonas*, matching SSA baseline performance and demonstrating that biochemical networks can support adaptive, cell-level information foraging.

Abstract

Living systems navigate environments using noisy and incomplete sensory signals. In unicellular algae, phototaxis is often modeled as a mechanistic run--tumble process driven by stimulus--response rules. However, such descriptions overlook how organisms actively sample their environment to reduce sensory ambiguity. From a minimal cognition perspective, we reframe this navigation as a subjective, information-driven sensorimotor process. To this end, we propose a framework linking a Partially Observable Markov Decision Process (POMDP) with biochemical reaction dynamics. Environmental variables are hidden, while the cell updates a minimal internal state from each observation through a memoryless Bayesian step. These internal dynamics balance orienting toward light with exploratory reorientation and can be implemented through Chemical-Reaction-Network Ordinary Differential Equations (CRN--ODEs). Our model includes a biophysical observation process for photoreception and a chemically computable polynomial bound on information gain. Using Inverse Reinforcement Learning (IRL) on 30 experimentally recorded Chlamydomonas trajectories, we infer the behavioral objective consistent with observed phototactic motion and benchmark the resulting dynamics with standard Stochastic Simulation Algorithm (SSA) baselines. Our model reproduces the empirical alignment-to-light distribution, comparable to objective SSA baselines on this dataset. Within this framework, run--tumble alternation emerges as an information-acquisition strategy: tumbling reorients the cell to sample new sensory configurations and resolve sensor ambiguity, demonstrating how intracellular biochemical networks can support adaptive information-seeking behavior in cellular navigation.

Problem

• phototaxis
• sensory ambiguity
• information-seeking behavior
• cellular navigation
• minimal cognition

Innovation

• reinforcement learning
• chemical reaction networks
• phototaxis
• information gain
• POMDP