🤖 AI Summary
This work addresses the challenges in no-reference image quality assessment arising from the difficulty of effectively fusing natural scene statistics (NSS) features with visual language model (VLM) embeddings, and the fact that the contribution of each feature varies dynamically across distortion types. To this end, the authors propose a distortion-aware dynamic fusion framework that adaptively weights 138-dimensional NSS features alongside SigLIP and CLIP-H embeddings via a multiplicative gating mechanism conditioned on input image content, without requiring fine-tuning of the VLM backbone. This approach achieves the first content-conditioned dynamic fusion of NSS and VLM features, with gating weights aligning closely with human distortion analyses. The method sets new state-of-the-art results on KonIQ-10k, KADID-10k, and LIVE Challenge, achieving an SROCC of 0.9715 and PLCC of 0.9733 on KADID-10k, and demonstrates that NSS features are most influential for noise and color distortion types.
📝 Abstract
Blind image quality assessment (BIQA) aims to predict perceived image quality without access to a reference image. Classical natural scene statistics (NSS) descriptors and modern vision-language model (VLM) embeddings address this problem from fundamentally different perspectives, yet whether combining them yields complementary benefits and how to weight their contributions per input image remains unexplored. We propose a distortion-aware fusion framework that integrates a 138-dimensional NSS descriptor with two complementary VLM embeddings, SigLIP and CLIP-H, through a multiplicative gating mechanism that learns per-input stream weights conditioned on image content. Unlike static concatenation fusion, the proposed gating network suppresses or amplifies each stream's contribution based on the input, producing weights that correlate positively (Spearman rank correlation rho=0.33) with the per-distortion NSS contribution measured by independent ablation on KADID-10k. The framework requires no end-to-end fine-tuning of the VLM backbones and is trained with a hybrid loss combining mean squared error, Pearson linear correlation, and pairwise ranking objectives. We evaluate on three standard benchmarks: KonIQ-10k (SROCC=0.9142, PLCC=0.9279), KADID-10k (SROCC=0.9715, PLCC=0.9733, surpassing recent state-of-the-art methods), and LIVE Challenge in-the-Wild (SROCC=0.8527, PLCC=0.8802 with cross-dataset pretraining and fine-tuning). A per-distortion analysis on KADID-10k reveals that NSS features contribute most on noise and color-shift distortions where pixel statistics are directly affected, and least on perceptual distortions such as color saturation changes. The learned gate values validate these findings, confirming that the model autonomously discovers distortion-stream affinity patterns consistent with the manual per-distortion study.