LL-Bench: Rethinking Low-Level Vision Evaluation in the Era of Large-Scale Generative Models

2026-06-01

AI Summary

Existing large-scale generative models lack systematic evaluation on low-level vision tasks that require pixel-level control. To address this gap, this work introduces LL-Bench, a comprehensive benchmark comprising 16 categories of real-world image degradations, 2,469 test images, and diverse restoration results, augmented with human preference judgments and quality ratings. The study further proposes LL-Score, the first MLLM-based evaluator explicitly aligned with human perception, which reveals significant discrepancies between conventional metrics and human judgments. Experimental results demonstrate that LL-Score outperforms existing image quality assessment metrics and serves as an effective reward signal for training generative models in low-level vision tasks.

Abstract

Large-scale generative models have demonstrated remarkable capabilities across image generation and editing tasks. However, their performance in low-level vision tasks, which require pixel-wise control, remains insufficiently studied. To address this gap, we introduce \textbf{LL-Bench}, a comprehensive \textbf{Benchmark} for evaluating the capabilities of large-scale generative models on \textbf{L}ow-\textbf{L}evel vision tasks. The benchmark comprises 2,469 real-world degraded images covering 16 low-level degradation tasks, and 28,919 restored images produced by 10 state-of-the-art large-scale generative models and 21 conventional restoration models, which are annotated with 152,020 expert-level pairwise human preferences and 28,334 quality scores. Built upon LL-Bench, we present a systematic diagnosis that reveals the performance boundaries and unique failure modes of large-scale generative models across diverse low-level vision tasks, compared with conventional representative restoration approaches. Moreover, we investigate the effectiveness of current quality evaluation metrics on LL-Bench, which exhibit significant discrepancy with human ratings. To better align restored-image quality assessment with human preferences, we further propose \textbf{LL-Score}, an MLLM-based evaluator that captures both restoration quality and hallucination existence. Extensive experiments demonstrate that LL-score not only outperforms existing image quality assessment metrics, but also serves as a promising reward model for training generative models on low-level vision tasks.

Problem

• low-level vision
• generative models
• image restoration
• quality assessment
• human preference

Innovation

• LL-Bench
• low-level vision
• generative models
• human preference alignment
• LL-Score