FAU at ImageCLEF 2026 Task on Multimodal Reasoning Robust Candidate Scoring and Concise Multilingual Visual Answering

2026-08-02

AI Summary

This work addresses the challenges posed by dense text, diagrams, and mathematical formulas in multilingual educational and scientific images for visual multiple-choice and open-ended question answering. The authors propose a zero-shot reasoning framework that avoids fine-tuning: for multiple-choice questions, it employs a candidate answer scoring mechanism based on vision-language model logits combined with an ensemble voting strategy; for open-ended questions, it integrates image enhancement, concise prompting, deterministic decoding, and targeted post-processing to suppress reasoning artifacts and formatting noise. Emphasizing the critical role of output control and post-processing in multimodal question answering, the method achieved third place in the multiple-choice subtask (accuracy: 0.7108) and first place in the open-ended subtask (top scores on COMET: 0.6488 and other metrics) at ImageCLEF 2026.

Abstract

We present our ImageCLEF 2026 Multimodal Reasoning system for the Visual Multiple Choice Question Answering (Visual MCQ) and Visual Open Question Answering (Visual OpenQA) subtasks. The challenge requires reliable reasoning over multilingual educational and scientific images with dense text, diagrams, charts, tables, formulas, and units, while enforcing strict answer formats. Our central finding is that robust output control is as important as model choice. For Visual MCQ, we replace fragile free-form generation with direct candidate label scoring from vision-language model logits, then combine complementary runs through score fusion and voting. For Visual OpenQA, we use image enhancement, concise final answer prompting, deterministic decoding, and targeted post-processing to remove reasoning traces and formatting artifacts. Without task-specific model training, our official submissions achieved third place in Visual MCQ with 0.7108 accuracy and first place in Visual OpenQA with 0.6488 COMET, 0.1391 BLEU, 0.2762 ROUGE L, and 0.2383 METEOR. The results highlight the practical value of inference engineering: careful scoring, ensembling, prompting, and cleanup can turn strong VLMs into reliable competition systems.

Problem

• Multimodal Reasoning
• Visual Question Answering
• Robust Candidate Scoring
• Concise Multilingual Answering
• Strict Answer Format

Innovation

• inference engineering
• candidate scoring
• deterministic decoding
• multimodal reasoning
• output control