Vision Language Models are Biased

2025-05-29

AI Summary

This work reveals that vision-language models (VLMs) suffer from systematic bias in objective visual tasks—such as counting and identification—due to interference from internet-derived prior knowledge, e.g., failing to correctly count stripes in a modified Adidas logo. To quantitatively characterize this knowledge bias for the first time, we propose a counterfactual image testing framework and an automated evaluation protocol: it leverages multi-domain counterfactual image generation, rigorous prompt control (e.g., “rely solely on the image”), and cross-domain bias benchmarking. Experiments span seven purely visual tasks; VLMs achieve only 17.05% average counting accuracy. Text injection further degrades performance, while image-prioritized prompting yields merely a 2-percentage-point improvement. This study establishes the first empirical foundation for prior-knowledge interference in VLM visual reasoning and introduces a reproducible, diagnostic paradigm for bias analysis.

Technology Category

• Computer Vision: Language and Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large language models (LLMs) memorize a vast amount of prior knowledge from the Internet that help them on downstream tasks but also may notoriously sway their outputs towards wrong or biased answers. In this work, we test how the knowledge about popular subjects hurt the accuracy of vision language models (VLMs) on standard, objective visual tasks of counting and identification. We find that state-of-the-art VLMs are strongly biased (e.g, unable to recognize a fourth stripe has been added to a 3-stripe Adidas logo) scoring an average of 17.05% accuracy in counting (e.g., counting stripes in an Adidas-like logo) across 7 diverse domains from animals, logos, chess, board games, optical illusions, to patterned grids. Insert text (e.g.,"Adidas") describing the subject name into the counterfactual image further decreases VLM accuracy. The biases in VLMs are so strong that instructing them to double-check their results or rely exclusively on image details to answer improves counting accuracy by only +2 points, on average. Our work presents an interesting failure mode in VLMs and an automated framework for testing VLM biases. Code and data are available at: vlmsarebiased.github.io.

Problem

• VLMs exhibit bias in counting and identification tasks
• Prior knowledge negatively impacts VLM accuracy on visual tasks
• VLMs struggle to correct biases even with explicit instructions

Innovation

• Testing VLM biases on visual tasks
• Analyzing impact of text on accuracy
• Proposing automated bias testing framework