Attend or Perish: Benchmarking Attention in Algorithmic Reasoning

2025-02-28

AI Summary

This work investigates the algorithmic generalization capability of Transformer models on unseen input/output domains—such as novel sequence lengths, numerical ranges, or data types—and examines whether their attention mechanisms support robust symbolic reasoning. To this end, we introduce the first infinite-domain algorithmic benchmark comprising six diverse tasks, and propose a novel evaluation paradigm that decouples algorithmic functionality from memorization effects. We further design an interpretable, verifiable, attention-driven correctness analysis framework, integrating attention map visualization with attribution-based diagnostics. Empirical results reveal pervasive failures in length extrapolation and systematic misalignment between attention patterns and required logical operations across mainstream models. All tasks, evaluation protocols, and analysis tools are open-sourced, establishing a standardized infrastructure for advancing research on algorithmic robustness and interpretability of large language models.

Technology Category

• Computer Vision: Visual Reasoning & Symbolic Representations
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Can transformers learn to perform algorithmic tasks reliably across previously unseen input/output domains? While pre-trained language models show solid accuracy on benchmarks incorporating algorithmic reasoning, assessing the reliability of these results necessitates an ability to cleanse models' functional capabilities from memorization. In this paper, we propose an algorithmic benchmark comprising six tasks of infinite input domains where we can also disentangle and trace the correct, robust algorithm necessary for the task. This allows us to assess (i) models' ability to extrapolate to unseen types of inputs, including new lengths, value ranges or input domains, but also (ii) to assess the robustness of the functional mechanism in recent models through the lens of their attention maps. We make the implementation of all our tasks and interoperability methods publicly available at https://github.com/michalspiegel/AttentionSpan .

Problem

• Assessing transformers' ability to extrapolate to unseen input domains.
• Disentangling models' functional capabilities from memorization effects.
• Evaluating robustness of attention mechanisms in algorithmic reasoning tasks.

Innovation

• Algorithmic benchmark with infinite input domains
• Disentangle and trace correct robust algorithms
• Assess models' extrapolation and attention robustness