Are language models aware of the road not taken? Token-level uncertainty and hidden state dynamics
This work investigates whether large language models (LLMs) implicitly represent unselected reasoning paths—i.e., “untraversed chains of thought”—during text generation. Method: We propose an uncertainty quantification framework based on dynamic analysis of hidden states: (i) decoding intermediate-layer activations to predict future token distributions, and (ii) conducting activation intervention experiments to assess hidden-state sensitivity to multi-path competition. Contribution/Results: We demonstrate that hidden states not only encode the current token decision but also explicitly embed the geometric structure of alternative reasoning paths in latent space. Crucially, the degree of uncertainty encoded in these states strongly correlates with model controllability—i.e., the ease with which generation can be steered toward desired outputs. This study provides the first empirical evidence for LLMs possessing “path awareness” and establishes a principled, interpretable linkage among hidden-state representations, predictive uncertainty, and generation controllability.