A flexible Bayesian non-parametric mixture model reveals multiple dependencies of swap errors in visual working memory

2025-05-02

AI Summary

This study investigates the cognitive origins and mechanisms of “swap errors” in visual working memory—specifically, whether they arise from encoding deficits, storage noise, or retrieval failure—and examines their dependence on cue and report features. Method: We develop a Bayesian nonparametric mixture model (BNS) that captures swap-error dependencies across dual feature dimensions in a data-driven manner, circumventing restrictive assumptions inherent in conventional modeling approaches. Contribution/Results: We identify a nonmonotonic modulation of swap errors along the report feature dimension—indicating a novel encoding deficit mechanism distinct from binding or cue-based errors. The cue similarity effect is robustly replicated across multiple datasets; moreover, we first demonstrate this nonmonotonic pattern in a motion-direction–location task, challenging prior accounts that attributed swap errors to a single underlying cause. These findings refine theoretical models of working memory by dissociating swap errors from classical binding failures and highlighting feature-specific encoding vulnerabilities.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Mixture of Experts (MoE)

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Human behavioural data in psychophysics has been used to elucidate the underlying mechanisms of many cognitive processes, such as attention, sensorimotor integration, and perceptual decision making. Visual working memory has particularly benefited from this approach: analyses of VWM errors have proven crucial for understanding VWM capacity and coding schemes, in turn constraining neural models of both. One poorly understood class of VWM errors are swap errors, whereby participants recall an uncued item from memory. Swap errors could arise from erroneous memory encoding, noisy storage, or errors at retrieval time - previous research has mostly implicated the latter two. However, these studies made strong a priori assumptions on the detailed mechanisms and/or parametric form of errors contributed by these sources. Here, we pursue a data-driven approach instead, introducing a Bayesian non-parametric mixture model of swap errors (BNS) which provides a flexible descriptive model of swapping behaviour, such that swaps are allowed to depend on both the probed and reported features of every stimulus item. We fit BNS to the trial-by-trial behaviour of human participants and show that it recapitulates the strong dependence of swaps on cue similarity in multiple datasets. Critically, BNS reveals that this dependence coexists with a non-monotonic modulation in the report feature dimension for a random dot motion direction-cued, location-reported dataset. The form of the modulation inferred by BNS opens new questions about the importance of memory encoding in causing swap errors in VWM, a distinct source to the previously suggested binding and cueing errors. Our analyses, combining qualitative comparisons of the highly interpretable BNS parameter structure with rigorous quantitative model comparison and recovery methods, show that previous interpretations of swap errors may have been incomplete.

Problem

• Understanding swap errors in visual working memory
• Exploring dependencies of swap errors on stimulus features
• Investigating encoding's role in swap errors versus retrieval

Innovation

• Bayesian non-parametric mixture model for swap errors
• Flexible dependency on probed and reported features
• Reveals non-monotonic modulation in report feature dimension