🤖 AI Summary
This work addresses the challenge of quantitatively evaluating latent capabilities—particularly harmful behaviors—in language models. We propose **conditional distance** as a unified metric, formalizing soft prompt optimization as the minimal semantic perturbation required to activate a target behavior—introducing the first principled formulation of *conditional behavioral reachability*. We design a novel **generalized conditional soft prompting** framework enabling consistent, cross-task (e.g., NLP, chess, grid pathfinding) and cross-domain evaluation. Integrating gradient-driven embedding optimization, conditional behavioral modeling, and an automated pipeline, our approach yields interpretable, comparable, and scalable latent capability probing. Experiments demonstrate that our method effectively quantifies the difficulty of behavior activation, providing red-teaming assessments with actionable, quantitative feedback.
📝 Abstract
To help evaluate and understand the latent capabilities of language models, this paper introduces an approach using optimized input embeddings, or 'soft prompts,' as a metric of conditional distance between a model and a target behavior. The technique aims to facilitate latent capability discovery as a part of automated red teaming/evaluation suites and to provide quantitative feedback about the accessibility of potentially concerning behaviors in a way that may scale to powerful future models, including those which may otherwise be capable of deceptive alignment. An evaluation framework using soft prompts is demonstrated in natural language, chess, and pathfinding, and the technique is extended with generalized conditional soft prompts to aid in constructing task evaluations.