Inducing Dyslexia in Vision Language Models

2025-09-29

AI Summary

Traditional behavioral and neuroimaging approaches struggle to establish causal mechanisms underlying developmental dyslexia. To address this, we propose the first functional computational simulation framework based on vision-language models (VLMs). Inspired by cognitive neuroscience, we design targeted stimulus paradigms and employ unit selectivity analysis coupled with selective ablation to precisely identify and perturb orthographic-processing modules within VLMs—inducing specific phonological deficits while preserving general multimodal capabilities. This intervention successfully dissociates phonological and orthographic processing pathways, recapitulates core cognitive phenotypes of dyslexia (e.g., impaired phoneme awareness and rapid naming), and causally validates the necessity of the visual word form area for reading. Our work establishes a novel, interventionist, interpretable, and reproducible computational paradigm for investigating dyslexia’s neural and cognitive mechanisms.

Technology Category

• Computer Vision: Language and Vision
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Dyslexia, a neurodevelopmental disorder characterized by persistent reading difficulties, is often linked to reduced activity of the visual word form area in the ventral occipito-temporal cortex. Traditional approaches to studying dyslexia, such as behavioral and neuroimaging methods, have provided valuable insights but remain limited in their ability to test causal hypotheses about the underlying mechanisms of reading impairments. In this study, we use large-scale vision-language models (VLMs) to simulate dyslexia by functionally identifying and perturbing artificial analogues of word processing. Using stimuli from cognitive neuroscience, we identify visual-word-form-selective units within VLMs and demonstrate that targeted ablation of these units, unlike ablation of random units, leads to selective impairments in reading tasks while general visual and language comprehension abilities remain intact. In particular, the resulting model matches dyslexic humans' phonological deficits without a significant change in orthographic processing. Taken together, our modeling results replicate key characteristics of dyslexia and establish a computational framework for investigating reading disorders.

Problem

• Simulating dyslexia in vision-language models by perturbing word processing units
• Investigating causal mechanisms of reading impairments through computational modeling
• Establishing artificial analogues to study neurodevelopmental reading disorders

Innovation

• Simulate dyslexia by perturbing word processing units
• Ablate visual-word-form units to impair reading selectively
• Replicate dyslexia traits while preserving other cognitive abilities