MAR-MAER: Metric-Aware and Ambiguity-Adaptive Autoregressive Image Generation

2026-04-02

AI Summary

Existing autoregressive image generation models suffer from limitations in generation quality, alignment with human preferences, and responsiveness to semantically ambiguous prompts. To address these issues, this work proposes a hierarchical autoregressive framework that integrates metric-aware embedding regularization, introduces controllable stochasticity via conditional variational autoencoding, and aligns generated outputs with human preferences through a lightweight projection head combined with an adaptive kernel regression loss. The proposed method achieves notable improvements of 1.6 in CLIPScore on the COCO benchmark and 5.3 in HPSv2 on the Ambiguous-Prompt Benchmark, demonstrating significant gains in image quality, diversity, and semantic coherence. These enhancements are consistently validated by both human evaluations and automatic metrics.

Abstract

Autoregressive (AR) models have demonstrated significant success in the realm of text-to-image generation. However, they usually face two major challenges. Firstly, the generated images may not always meet the quality standards expected by humans. Furthermore, these models face difficulty when dealing with ambiguous prompts that could be interpreted in several valid ways. To address these issues, we introduce MAR-MAER, an innovative hierarchical autoregressive framework. It combines two main components. It is a metric-aware embedding regularization method. The other one is a probabilistic latent model used for handling ambiguous semantics. Our method utilizes a lightweight projection head, which is trained with an adaptive kernel regression loss function. This aligns the model's internal representations with human-preferred quality metrics, such as CLIPScore and HPSv2. As a result, the embedding space that is learned more accurately reflects human judgment. We are also introducing a conditional variational module. This approach incorporates an aspect of controlled randomness within the hierarchical token generation process. This capability allows the model to produce a diverse array of coherent images based on ambiguous or open-ended prompts. We conducted extensive experiments using COCO and a newly developed Ambiguous-Prompt Benchmark. The results show that MAR-MAER achieves excellent performance in both metric consistency and semantic flexibility. It exceeds the baseline Hi-MAR model's performance, showing an improvement of +1.6 in CLIPScore and +5.3 in HPSv2. For unclear inputs, it produces a notably wider range of outputs. These findings have been confirmed through both human evaluation and automated metrics.

Problem

• autoregressive image generation
• image quality
• ambiguous prompts
• metric consistency
• semantic flexibility

Innovation

• metric-aware regularization
• ambiguity-adaptive generation
• autoregressive image synthesis
• conditional variational module
• embedding alignment