Generative Modeling of Discrete Latent Structures via Dynamic Policy Gradients

2026-06-05

AI Summary

Inferring compositional discrete mechanistic latent states from indirect observations is highly challenging: conventional methods struggle to scale to large combinatorial spaces, while existing deep learning approaches often produce implausible latent configurations. To address this, this work proposes GReinSS, a novel framework that uniquely integrates policy gradient optimization with dynamic reward rescaling and variational generative modeling. By optimizing the latent state distribution to maximize observational likelihood, GReinSS effectively recovers authentic discrete latent structures. The method achieves substantially higher reconstruction accuracy than baseline approaches on simulated latent set and latent graph tasks. Furthermore, when applied to RNA-seq short-read data, GReinSS reconstructs transcript isoforms that exhibit greater consistency with long-read sequencing results, outperforming established algorithms such as RSEM.

Abstract

Many scientific problems require inferring unobserved mechanistic latent states from indirect observations. While classical approaches, including expectation maximization, do not scale to combinatorially large spaces, deep learning approaches such as variational autoencoders typically form artificial latent states rather than reconstructing the mechanistic ground-truth states. Here, we introduce GReinSS, a policy learning framework that uses dynamically rescaled rewards to learn latent state distributions that maximize the observed data likelihood. We show that GReinSS accurately reconstructs simulated latent sets and latent graphs, outperforming alternative policy learning and generative modeling baselines. Additionally, GReinSS reconstructs isoforms from real short-read RNA sequencing data that better match isoforms detected by orthogonal long-read sequencing than the standard RSEM algorithm. Overall, GReinSS is a principled and practically effective approach for generative modeling and inference of combinatorial latent states from indirect observations.

Problem

• discrete latent structures
• generative modeling
• indirect observations
• combinatorial latent states
• mechanistic inference

Innovation

• discrete latent structures
• dynamic policy gradients
• generative modeling
• combinatorial inference
• RNA isoform reconstruction