Reading and Steering Representations of Materials-Science Mechanisms in an Open-Weight Language Model

2026-07-22

AI Summary

Although large language models can answer scientific questions, it remains unclear whether they genuinely represent and utilize physical mechanisms. This work addresses this gap in the domain of materials science by proposing an integrated probing framework that combines interpretability analysis, state geometry, causal interventions, and counterfactual benchmarks. For the first time, it disentangles three forms of mechanistic representation: conceptual interpretability, constitutive directionality, and causal control. Leveraging matched direct and Jacobian-based lexical readouts, option-free state geometry, counterfactual benchmarks spanning 60 physical laws, and bidirectional causal interventions, the framework successfully reconstructs concept orderings, identifies 9 out of 10 mechanism families in blind tests, correctly infers the directionality of 39 out of 40 directional laws, and significantly modulates model output probabilities—thereby revealing how physical principles are internally encoded within the model.

Abstract

Large language models can answer scientific questions, yet a correct output does not reveal whether the model represents or uses the governing physics. Here we show that materials science mechanism information in the open-weight google/gemma-4-E4B-it model has three experimentally separable forms: concepts are readable in individual hidden states, constitutive orientation is carried by controlled transformations between states, and selected internal representations causally control engineering answers. We combine matched direct and Jacobian vocabulary readouts, option-free state geometry, a 60-law counterfactual benchmark and causal interventions. In 50 held-out materials descriptions, three independently fitted Jacobian lenses reproduced concept ranks, and target-free word sets from both readouts enabled blinded identification of 9 of 10 mechanism families. A separate 72-prompt benchmark produced mechanism-specific hidden-state neighborhoods, but an exact graph audit showed that this apparent physical organization was equally explained by numerical comparison. We therefore compared otherwise identical prompts in which only the direction of the physical input was reversed, asking whether the resulting hidden-state movement followed the supplied constitutive law. These state transformations ordered direct, physically neutral and inverse laws across 60 frozen relations and correctly oriented 39 of 40 directional laws, whereas lexical controls were near chance. Bidirectional interventions shifted answer probabilities toward or away from the physically appropriate outcome across all 12 matched cases, while counterfactual state patches transferred opposing decision signals across mechanisms and answer formats. Physical relationships were therefore more visible in controlled state changes than in absolute states alone.

Problem

• language models
• materials science
• mechanism representation
• physical reasoning
• model interpretability

Innovation

• causal intervention
• constitutive laws
• hidden-state geometry
• counterfactual benchmark
• Jacobian readout