Harnessing Vision-Language Models for Perceptual Quality Assessment and Autonomous Content Adjustment in Augmented Reality

2026-09-30

AI Summary

This study addresses the reliance on costly, labor-intensive, and poorly scalable manual evaluation in augmented reality (AR) visual quality assessment by proposing an automated framework based on vision-language models (VLMs). Methodologically, we construct the RateAR benchmark dataset and design a context-aware prompting strategy to enable precise prediction of AR perceptual quality and content-adaptive optimization without human intervention. Experimental results demonstrate that VLM predictions achieve a correlation of 0.87 with subjective human ratings. Furthermore, over 90% of users confirm that the system effectively enhances the consistency of virtual content placement. Overall, this work establishes an efficient and scalable new paradigm for AR quality assessment.

Abstract

Advancements in augmented reality (AR) continue to foster innovative solutions, facilitating novel methodologies within educational systems, healthcare delivery, and risk-mitigation protocols. However, optimizing for end-user immersion and comfort remains challenging, as AR head-mounted displays contend with constrained scene geometry, spatial jitter, and temporal instability. User studies are the standard AR evaluation method for visual quality, but their cost, diminishing scalability, and inflexibility pose bottlenecks during iterative application design. To address this problem, we present an automated framework for AR content evaluation and refinement, built on vision-language models (VLMs), to evaluate and predict the visual fidelity of AR scenes as perceived by users. First, we introduce RateAR, a benchmark of AR images and videos collected across diverse scenes and environmental conditions, with good-to-excellent reliability (ICC(2,5) >= .90) across perceptual factors, including object placement, scale, and shadow consistency. Subsequently, we evaluate eleven commercial VLMs on the crafted benchmark. Results support that VLM-based quality predictions strongly correlate with human subjective judgments, achieving Spearman's rank-order correlations of up to 0.8695. An ablation study further suggests that, compared to other prompting strategies, our contextual prompting yields better alignment with human ratings while balancing introduced complexity cues. Building on these findings, we construct an automated AR content adjustment system and conduct a 21-participant user study. More than 90% of participants found that the system improved placement and size coherence of virtual content.

Problem

• Augmented Reality
• Perceptual Quality Assessment
• Vision-Language Models
• Content Adjustment
• Visual Fidelity

Innovation

• Vision-Language Models
• Augmented Reality
• Perceptual Quality Assessment
• Contextual Prompting
• Automated Content Adjustment