image quality assessment

Accurately measuring visual fidelity and predicting human perception remains challenging, particularly in distinguishing perceptual differences across object categories and simplification types. Method: We propose a systematic evaluation framework integrating polygon mesh simplification (using two distinct algorithms) with psychophysical experiments—namely naming time, subjective rating, and pairwise preference judgments—to rigorously compare perceptual responses to animals versus man-made objects under controlled simplification levels. Contribution/Results: This work introduces the first three-factor controlled analysis jointly varying simplification type, degree, and object category. We find naming time and preference tasks exhibit higher sensitivity to fidelity degradation than subjective ratings; conversely, state-of-the-art image- and mesh-based automatic metrics reliably predict only subjective ratings, failing to model higher-order perceptual decisions. These findings expose fundamental cognitive limitations of prevailing fidelity metrics and provide empirical grounding and a methodological paradigm for developing human-centered graphics fidelity assessment frameworks.

This study investigates whether mainstream image/video quality metrics—such as SSIM, LPIPS, and VMAF—faithfully model human low-level visual mechanisms, including contrast sensitivity, contrast masking, and contrast matching. To this end, we introduce the first interpretability benchmark framework specifically designed for low-level vision properties, grounded in psychophysical principles: contrast sensitivity function testing, masking threshold estimation, and contrast-matching discrimination tasks. We systematically evaluate 33 full-reference metrics under this framework. Results show that LPIPS and MS-SSIM effectively capture contrast masking, whereas VMAF exhibits significant deficiencies; SSIM over-responds to high-frequency distortions, while MS-SSIM substantially mitigates this bias. Our analysis uncovers structural shortcomings in current metrics’ perceptual modeling, revealing misalignments with neurobiologically grounded vision principles. This work establishes a new paradigm for interpretable, neurophysiologically plausible quality assessment and provides empirical foundations for next-generation perceptual metrics.

Existing 3D scene evaluation metrics predominantly rely on holistic image-quality measures, exhibiting substantial misalignment with human visual perception. To address this, we propose Objectness SIMilarity (OSIM), the first object-level assessment metric grounded in *objectness*—a cognitive principle reflecting human perceptual prioritization of coherent, semantically meaningful entities. OSIM leverages pre-trained object detectors to extract instance-aware features, then quantifies fidelity via neurocognitively inspired objectness modeling and cross-scene feature similarity computation, enabling end-to-end object-level evaluation. Its core innovations are: (i) shifting evaluation granularity from pixel- or image-level to object-level, and (ii) introducing cognition-driven objectness as a foundational concept for perceptual assessment. Extensive experiments demonstrate that OSIM achieves significantly higher correlation with human subjective ratings than PSNR, LPIPS, and DINO. When re-evaluating twelve state-of-the-art 3D reconstruction and generation models under a unified benchmark, OSIM rankings better reflect perceptual realism.

Existing super-resolution (SR) models improve visual quality but often introduce high-level semantic hallucinations, causing content distortion; conventional low-level metrics (e.g., PSNR, LPIPS) lack sensitivity to such semantic inconsistencies. Method: This work formally defines and quantifies the high-level semantic fidelity problem in SR, introduces FID-SR—the first large-scale benchmark with human-annotated fidelity scores—and reveals the weak correlation between mainstream metrics and semantic consistency. We further propose a multimodal foundation model–based fidelity evaluator and integrate it into SR model fine-tuning via fidelity-aware feedback. Contribution/Results: Empirical analysis shows that prevalent SR models suffer from widespread semantic distortion. Incorporating fidelity feedback enables models to preserve visual quality while significantly improving semantic consistency, validating the effectiveness and practicality of the proposed evaluation paradigm.

This work proposes a novel full-reference image quality assessment (IQA) paradigm centered on machine task utility rather than human perception. To this end, the authors introduce “latent machine utility” as a new evaluation criterion and develop ML-CLIPSim, a multi-layer similarity metric built upon a frozen, pre-trained CLIP visual encoder. ML-CLIPSim jointly leverages consistency between intermediate patch tokens and global embeddings across layers and constructs the PCMP dataset via predicted consistency voting. Experiments demonstrate that the proposed approach consistently outperforms conventional fidelity- and perception-based metrics across diverse benchmarks—including machine preference evaluation, human IQA datasets, and learned image compression—significantly improving the rate–task performance trade-off.

This work addresses the limitation of existing image perceptual similarity metrics in capturing the context-dependent nature of human judgments across diverse semantic dimensions—such as shape or color. To this end, the authors construct a large-scale dataset of human triplet similarity judgments annotated with free-form semantic dimensions and leverage it to fine-tune state-of-the-art vision-language models. They propose TPIPS (Text-Prompted Perceptual Image Similarity), a novel metric that enables dynamic specification of similarity semantics via natural language prompts, thereby overcoming the rigidity of conventional single-purpose similarity measures. Experiments demonstrate that TPIPS significantly outperforms existing methods in aligning with human judgments and generalizing across data distributions. The approach also proves effective in downstream applications including text-guided image retrieval, compositional search, and fine-grained evaluation of generative models.

In most real-world image-to-image (I2I) scenarios, existing evaluations primarily focus on instruction following and the perceptual quality or aesthetics of the generated images. However, they largely fail to assess whether the output image preserves the semantic correspondence and spatial structure of the input image. To address this limitation, we propose StableI2I, a unified and dynamic evaluation framework that explicitly measures content fidelity and pre--post consistency across a wide range of I2I tasks without requiring reference images, including image editing and image restoration. In addition, we construct StableI2I-Bench, a benchmark designed to systematically evaluate the accuracy of MLLMs on such fidelity and consistency assessment tasks. Extensive experimental results demonstrate that StableI2I provides accurate, fine-grained, and interpretable evaluations of content fidelity and consistency, with strong correlations to human subjective judgments. Our framework serves as a practical and reliable evaluation tool for diagnosing content consistency and benchmarking model performance in real-world I2I systems.

This work addresses the limitation of existing image quality assessment methods, which primarily focus on global distortions and struggle to effectively detect localized perceptual artifacts such as ghosting, lens flare, and moiré patterns. To this end, we introduce the Image Perceptual Artifact Detection (IPAD) task for the first time, along with a benchmark dataset comprising both real-world and synthetically generated samples. We propose the IPAD-CLIP framework, which leverages artifact-aware textual embeddings to guide the CLIP vision encoder toward subtle local anomalies while modeling semantic context to capture artifact-object relationships. Our approach significantly outperforms current anomaly and tampering detection methods on the proposed benchmark, achieving superior detection performance with high computational efficiency.

Existing evaluation metrics for image and video object removal often misalign with human perception, frequently favoring results that rely on copy-pasting, exhibit blurriness, or overlook local artifacts. To address this limitation, this work proposes a perceptually aligned Removal Consistency (RC) evaluation framework, comprising a spatial consistency metric (RC-S) based on sliding-window feature comparisons and a temporal consistency metric (RC-T) that tracks shared region distributions across frames. The authors also introduce PROVE-Bench, a comprehensive benchmark encompassing both paired and unpaired subsets to reflect real-world scenarios and challenging cases. Extensive experiments demonstrate that RC significantly outperforms existing metrics across multiple benchmarks and achieves high correlation with human judgments. The code and dataset are publicly released.