Coupled Continuous-Discrete Generation for Scene Text Image Super-Resolution

2026-08-05

AI Summary

This work addresses the challenge of jointly preserving character semantics and restoring visual quality in scene text image super-resolution by proposing DualTSR, a unified framework that, for the first time, couples continuous image generation with discrete text generation within an end-to-end architecture. DualTSR recovers image latent variables via conditional flow matching while reconstructing text tokens through absorbing-state discrete diffusion, enabling full cross-modal interaction throughout the process via a shared multimodal Transformer—without relying on external OCR priors. Evaluated on CTR-TSR and RealCE benchmarks, DualTSR substantially outperforms existing methods, achieving a 12.78% absolute gain in X4 super-resolution accuracy over DiffTSR, reducing model size from 1.23B to 203M parameters, and decreasing inference latency from 13.3 seconds to 132 milliseconds.

Abstract

Scene text image super-resolution (STISR) aims to recover visually plausible appearance while preserving character semantics from degraded inputs. Existing STISR systems often rely on externally generated priors or separate image and text models, resulting in error propagation and costly multi-stage inference. We present DualTSR, a unified framework that formulates STISR as coupled continuous-discrete generation. Conditional flow matching restores continuous image latents, while absorbing-state discrete diffusion reconstructs text tokens. Both processes share a multimodal transformer backbone, allowing the evolving image and text states to interact throughout generation without an external OCR prior at inference. On CTR-TSR, DualTSR achieves the best FID, LPIPS, ACC, and NED among the compared methods at both X2 and X4. On an aligned RealCE subset, it obtains the best FID, ACC, and NED with competitive LPIPS. Compared with DiffTSR at X4, DualTSR improves ACC by 12.78 percentage points while reducing the parameter count from 1.23B to 203M and end-to-end latency from 13.3s to 132ms. These results establish DualTSR as an accurate and efficient method for STISR.

Problem

• Scene Text Image Super-Resolution
• Continuous-Discrete Generation
• Multimodal Interaction
• Error Propagation
• Efficient Inference

Innovation

• coupled continuous-discrete generation
• conditional flow matching
• absorbing-state discrete diffusion
• multimodal transformer
• scene text image super-resolution