From Plausibility to Verifiability: Risk-Controlled Generative OCR for Vision-Language Models

2026-03-20

AI Summary

This work addresses a critical limitation in generative OCR systems based on frozen vision-language models, where autoregressive decoding often prioritizes semantic plausibility over visual and geometric evidence, leading to high-risk errors such as hallucinated or unsupported character substitutions. To mitigate this, the authors propose a model-agnostic geometric risk control framework that reframes OCR as a selective accept/reject task. By leveraging multi-view input probing, structured filtering, cross-view consistency checks, and stability criteria, the framework outputs predictions only when geometric verifiability conditions are satisfied. Evaluated on standard OCR benchmarks, the approach substantially reduces catastrophic error rates while maintaining controllable coverage trade-offs, enabling more reliable deployment and advancing generative OCR toward geometrically grounded, rather than merely semantically plausible, outputs.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Modern vision-language models (VLMs) can act as generative OCR engines, yet open-ended decoding can expose rare but consequential failures. We identify a core deployment misalignment in generative OCR. Autoregressive decoding favors semantic plausibility, whereas OCR requires outputs that are visually grounded and geometrically verifiable. This mismatch produces severe errors, especially over-generation and unsupported substitutions, creating deployment risk even when benchmark accuracy remains high. We therefore formulate frozen VLM OCR as a selective accept/abstain problem and propose a model-agnostic Geometric Risk Controller. The controller probes multiple structured views of the same input, applies lightweight structural screening, and accepts a transcription only when cross-view consensus and stability satisfy predefined criteria, yielding a small family of operating points. Experiments on frozen VLM backbones and standard OCR benchmarks show consistent reductions in extreme-error risk and catastrophic over-generation at predictable coverage costs. Reliable deployment of generative OCR with frozen VLMs benefits from explicit system-level risk control rather than unconstrained generation.

Problem

• generative OCR
• vision-language models
• verifiability
• deployment risk
• over-generation

Innovation

• Generative OCR
• Vision-Language Models
• Risk Control
• Geometric Verification
• Selective Accept/Abstain