LLM Context Conditioning and PWP Prompting for Multimodal Validation of Chemical Formulas

2025-05-18

AI Summary

Large language models (LLMs) exhibit limited capability in detecting subtle technical errors—particularly formulae embedded in scientific document images—and their inherent correction bias often obscures genuine errors. Method: We propose Persistent Workflow Prompting (PWP), a structured contextual modulation technique that, for the first time, leverages PWP to explicitly regulate LLM reasoning patterns. Using only standard chat interfaces—without API access or model modification—we guide general-purpose multimodal LLMs (e.g., Gemini 2.5 Pro, ChatGPT Plus o3) to perform text-image co-verification. Contribution/Results: Experiments demonstrate that Gemini 2.5 Pro identifies chemical formula errors in figures missed by human reviewers; ChatGPT Plus o3 achieves significant improvement in textual error detection. Our approach establishes a novel zero-shot, fine-tuning-free paradigm for enhancing LLM reliability in scientific document validation.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)Computer Vision: Diffusion Models for Vision

Application Category

Economics, Online Markets and Human Computation: LLM based quality controls for crowd workSearch and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved informationGraph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Identifying subtle technical errors within complex scientific and technical documents, especially those requiring multimodal interpretation (e.g., formulas in images), presents a significant hurdle for Large Language Models (LLMs) whose inherent error-correction tendencies can mask inaccuracies. This exploratory proof-of-concept (PoC) study investigates structured LLM context conditioning, informed by Persistent Workflow Prompting (PWP) principles, as a methodological strategy to modulate this LLM behavior at inference time. The approach is designed to enhance the reliability of readily available, general-purpose LLMs (specifically Gemini 2.5 Pro and ChatGPT Plus o3) for precise validation tasks, crucially relying only on their standard chat interfaces without API access or model modifications. To explore this methodology, we focused on validating chemical formulas within a single, complex test paper with known textual and image-based errors. Several prompting strategies were evaluated: while basic prompts proved unreliable, an approach adapting PWP structures to rigorously condition the LLM's analytical mindset appeared to improve textual error identification with both models. Notably, this method also guided Gemini 2.5 Pro to repeatedly identify a subtle image-based formula error previously overlooked during manual review, a task where ChatGPT Plus o3 failed in our tests. These preliminary findings highlight specific LLM operational modes that impede detail-oriented validation and suggest that PWP-informed context conditioning offers a promising and highly accessible technique for developing more robust LLM-driven analytical workflows, particularly for tasks requiring meticulous error detection in scientific and technical documents. Extensive validation beyond this limited PoC is necessary to ascertain broader applicability.

Problem

• Identifying subtle errors in scientific documents with multimodal content
• Modulating LLM behavior for precise validation without API access
• Improving error detection in chemical formulas using structured prompting

Innovation

• LLM context conditioning for error detection
• Persistent Workflow Prompting (PWP) principles
• Standard chat interfaces without API modifications