Scientific Image Quality Assessment via Multi-modal Retrieval-Augmented Generation

2026-09-16

AI Summary

本文提出了一种基于多模态检索增强生成的框架，通过整合文本语义与细粒度视觉特征来解决科学图像质量评估问题，并在SIQA挑战赛中取得优异成绩。

Abstract

This paper proposes a Retrieval-Augmented Generation (RAG) framework for scientific image quality assessment, designed to simultaneously address both the understanding track (SIQA-U) and the scoring track (SIQA-S) of the SIQA challenge. We construct a multimodal index that integrates textual semantics with fine-grained visual features, and develop a multi-route retrieval and fusion mechanism to provide large language models with highly relevant reference cases, thereby enhancing their capability to evaluate complex scientific images. Experimental results demonstrate that the proposed framework effectively aligns with the judgment criteria of human experts. Ultimately, our method achieves 1st place in the SIQA-U track of the SIQA challenge at the ICME 2026 Grand Challenges.

Problem

• Scientific Image Quality Assessment
• Retrieval-Augmented Generation
• Multimodal Index

Innovation

• Retrieval-Augmented Generation (RAG)
• multimodal index
• multi-route retrieval and fusion mechanism