Problem
Research questions and friction points this paper is trying to address.
Large Language Models
Probabilistic Safety Bounds
Harmful Output
Statistical Certification
PAC Bounds
Sound Probabilistic Safety Bounds for Large Language Models
🤖 AI Summary
This work proposes the first formally verifiable framework for estimating Probably Approximately Correct (PAC) safety lower bounds on the probability that large language models generate harmful content. By integrating Clopper-Pearson confidence intervals with a latent-space feature-guided strategy for exploring autoregressive generation trees, the method enables efficient and rigorous statistical certification even in regimes of extremely low harm probabilities. Experimental results demonstrate that the framework yields non-trivial and reliable safety lower bounds across mainstream large language models, substantially enhancing both the theoretical rigor and practical feasibility of safety evaluation for generative AI systems.
📝 Abstract
We propose a novel framework for computing rigorous bounds on the probability that a large language model (LLM) generates harmful output to a given prompt. We study a new application of the Clopper-Pearson confidence intervals to obtain probably approximately correct (PAC) bounds for this problem. As our main technical contribution, we propose an algorithm that leverages features in the latent space to prioritize exploring branches in the auto-regressive generation tree that are more likely to produce harmful outputs. Our approach in particular enables the efficient computation of useful lower bounds, even in scenarios where the true harm probability is extremely small, and crucially, the obtained lower bounds are sound, i.e., formally proven to be less than the actual harmfulness probability: our experimental results demonstrate the effectiveness of our method by computing non-trivial lower bounds on state-of-the-art LLMs. This study newly enables the evaluation and statistical certification of LLMs.
Innovation
Methods, ideas, or system contributions that make the work stand out.
probabilistic safety bounds
Clopper-Pearson confidence intervals
latent space guidance
sound lower bounds
LLM safety certification
🔎 Similar Papers
No similar papers found.
