Generative modeling of intrinsically disordered protein regions by reinforcing sparse autoencoder features

2026-10-01

AI Summary

This study addresses the challenges of functional design for intrinsically disordered proteins (IDPs) and the inherent bias of existing models toward folded domains by proposing IDiom, an autoregressive language model coupled with a reinforcement learning-based sparse autoencoder (RL-SAE) post-training method. Leveraging data mining from the AlphaFold database, this approach employs reinforcement learning to optimize sparse autoencoder features, enabling interpretable and composable control over sequence representations for function-directed generation. Experimental results demonstrate that the method achieves a 90% target feature activation rate and significantly outperforms conventional guided approaches in subcellular localization prediction and transcriptional activity optimization. By facilitating precise, controllable sequence design, this work establishes a novel paradigm for the rational engineering of IDPs.

Abstract

Intrinsically disordered protein regions (IDRs) play central roles in cellular processes such as transcriptional regulation, signal transduction, and subcellular localization, yet their functional design remains challenging. Structure-based design methods do not readily apply to IDRs, and existing protein language models are trained on full-length protein sequences, thus learning a prior that is biased towards folded domains. Here, we present IDiom, an autoregressive protein language model trained on IDiom-DB, a dataset of 54 million predicted IDRs curated from the AlphaFold Database. IDiom generates diverse sequences that recapitulate the composition, patterning, motifs, and predicted disorder of natural IDRs. To control function-associated sequence patterns, we also introduce reinforcement learning with sparse autoencoder features (RL-SAE), a post-training method that rewards the generation of sequences that activate specified feature sets. Across eight IDR design tasks, RL-SAE sequences activate, on average, 90% of 30 targeted features, compared to 24% for activation steering. We demonstrate that RL-SAE improves the predicted subcellular localization and transcriptional activity of generated IDRs compared to steering and supervised fine-tuning, and enables features associated with distinct biological functions to be combined within individual sequences. Thus, IDiom and RL-SAE enable interpretable and composable IDR design through explicit control of function-associated sequence features. More broadly, RL-SAE could extend to other protein design settings where interpretable features provide useful design targets. Code is available at https://github.com/rotskoff-group/idiom.

Problem

• intrinsically disordered regions
• protein design
• generative modeling
• protein language models

Innovation

• Intrinsically disordered regions
• Protein language model
• Reinforcement learning
• Sparse autoencoder
• Protein design