Probing Materials Knowledge in LLMs: From Latent Embeddings to Reliable Predictions

2026-03-02

AI Summary

This study investigates the reliability disparity of large language models (LLMs) in symbolic versus numerical tasks within materials science. Through systematic evaluation of 25 LLMs across four task categories—augmented with fine-tuning, intermediate-layer embedding extraction, response entropy analysis, and an 18-month longitudinal tracking of GPT-series models—the work uncovers a “LLM output head bottleneck”: while fine-tuning enhances consistency and verifiability in symbolic tasks, numerical reasoning remains prone to inconsistency. Notably, leveraging intermediate-layer embeddings yields significantly superior performance compared to conventional text-based outputs. Longitudinal analysis reveals performance fluctuations of 9–43% in GPT models, underscoring substantial reproducibility risks in scientific applications. This work presents the first demonstration of embedding-enhanced numerical regression and long-term performance monitoring of LLMs in materials science.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Symbolic Representations

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Large language models are increasingly applied to materials science, yet fundamental questions remain about their reliability and knowledge encoding. Evaluating 25 LLMs across four materials science tasks -- over 200 base and fine-tuned configurations -- we find that output modality fundamentally determines model behavior. For symbolic tasks, fine-tuning converges to consistent, verifiable answers with reduced response entropy, while for numerical tasks, fine-tuning improves prediction accuracy but models remain inconsistent across repeated inference runs, limiting their reliability as quantitative predictors. For numerical regression, we find that better performance can be obtained by extracting embeddings directly from intermediate transformer layers than from model text output, revealing an ``LLM head bottleneck,'' though this effect is property- and dataset-dependent. Finally, we present a longitudinal study of GPT model performance in materials science, tracking four models over 18 months and observing 9--43\% performance variation that poses reproducibility challenges for scientific applications.

Problem

• reliability
• knowledge encoding
• numerical prediction
• reproducibility
• large language models

Innovation

• large language models
• materials science
• embedding extraction
• numerical regression
• model reproducibility