Stockmark-Nemotron-3-Nano-Omni-JapanDocReader: Structured Document Parsing via Capability Injection and Forgetting Control

2026-08-06

AI Summary

This work addresses the challenge of catastrophic forgetting in document visual question answering (VQA) when injecting Japanese structured document parsing capabilities into multimodal reasoning models. Building upon Nemotron-3-Nano-Omni, the authors propose a parsing-oriented hybrid supervised fine-tuning framework synergistically combined with DAPO reinforcement learning, augmented by a variance-aware prompt filtering mechanism. A dual-stream synthetic data engine—comprising a Japanese document VQA stream and a procedural structured parsing stream—enables joint training that substantially enhances structured parsing performance beyond the pure supervised fine-tuning upper bound while effectively preserving the model’s original VQA proficiency. This approach achieves a balanced integration of new parsing skills without compromising pre-existing capabilities.

Abstract

We present Stockmark-Nemotron-3-Nano-Omni-JapanDocReader, a Japanese document understanding model built from Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16. The central goal of this work is structured document parsing via capability injection and forgetting control: we inject Japanese structured document parsing capability into a reasoning-oriented multimodal model while preserving its document VQA capability as much as possible. We study parsing-centric SFT, which uses only structured document parsing data; mixed SFT, which combines structured document parsing and VQA data; and parsing-centric RL, which optimizes structured parsing with a task-level reward. Our experiments show that parsing-centric SFT substantially improves structured document parsing performance but causes measurable VQA forgetting. Mixed SFT mitigates this forgetting while preserving nearly the same structured parsing performance. Applying DAPO-based parsing-centric RL on top of the mixed SFT checkpoint further improves structured document parsing beyond the SFT ceiling, producing the final released model. The training data is constructed with a data engine consisting of two complementary synthetic streams: a Japanese Document VQA Stream and a programmatic structured document parsing stream. We also discuss reward design and variance-based prompt filtering for continuous structured document parsing rewards, highlighting their importance for making RL effective in long-reasoning structured document parsing tasks.

Problem

• structured document parsing
• capability injection
• forgetting control
• document VQA
• Japanese document understanding

Innovation

• capability injection
• forgetting control
• structured document parsing
• DAPO-based RL
• synthetic data engine