BOReFT: Manifold Steering of Language Models for Black-box Optimization

2026-09-27

AI Summary

This study addresses the limited exploration space and the lack of a domain adapted for continuous optimization in black-box search over large language models (LLMs). We propose learning a low-dimensional latent intervention space within a frozen LLM, which serves as a continuous search domain for Bayesian optimization. By leveraging manifold guidance and latent state interventions, this approach establishes a principled bridge between discrete LLM proposals and continuous black-box optimization. Furthermore, we theoretically derive optimal score bounds under semantic coverage control. Experimental evaluations on Semantle and molecular design tasks demonstrate that the proposed method significantly outperforms existing baselines, substantially increasing both the number of discovered targets and their attribute scores.

Abstract

Language models are increasingly used as proposal models for black-box search, from program optimization to molecular design. Existing approaches typically improve proposals through iterative prompting or parameter updates, offering limited control over how completely and efficiently the model's search space is explored. Continuous optimization methods, such as Bayesian optimization, provide a principled way to search but require a suitable domain to operate over. To address this, we introduce BOReFT, which learns a compact, low-dimensional space of hidden-state interventions in a frozen language model, and uses this space as the search domain for Bayesian optimization with an external scoring function. Empirically, we find that the learned domain spans semantic regions and exhibits smoothness properties that support search. Theoretically, we show that semantic coverage and interpolation control the best score available in the learned space, and that decoding from this space yields a standard stochastic-bandit observation model for adaptive search. We evaluate BOReFT on the interpretable word search task"Semantle"and on three more real-world discovery tasks in de novo molecule property optimization. Compared to strong LLM baselines, BOReFT finds in Semantle a higher number of hidden targets and, on two out of three molecular objectives, achieves higher property scores. Consequently, our method provides a principled new bridge between discrete proposal spaces of LLM-based search and continuous black-box optimization.

Problem

• Black-box Optimization
• Language Models
• Bayesian Optimization
• Search Space Exploration
• Molecular Design

Innovation

• Black-box Optimization
• Bayesian Optimization
• Manifold Steering
• Hidden-state Interventions
• Language Models