Re-examining learning linear functions in context

2024-11-18

arXiv.org

AI Summary

This work investigates how large language models (LLMs) acquire and generalize simple mathematical relations—specifically univariate linear functions—via in-context learning (ICL). Method: Using from-scratch trained GPT-2–style Transformer models and rigorously controlled synthetic data, we systematically evaluate in-distribution generalization and out-of-distribution (OOD) induction. Contribution/Results: We find that models fail to abstract the underlying linear rule and do not perform implicit algorithmic reasoning (e.g., linear regression); instead, they capture only superficial statistical regularities. Critically, they exhibit complete failure on OOD test cases, confirming that ICL operates via memory-driven, local pattern matching rather than genuine functional induction. We propose the first mathematically precise, empirically verifiable hypothesis challenging dominant algorithmic-reasoning accounts of ICL. Our study establishes a novel, rigorous evaluation paradigm for generalization on mathematical tasks, providing both theoretical grounding and an empirical framework for characterizing fundamental limitations of LLM reasoning.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

In-context learning (ICL) has emerged as a powerful paradigm for easily adapting Large Language Models (LLMs) to various tasks. However, our understanding of how ICL works remains limited. We explore a simple model of ICL in a controlled setup with synthetic training data to investigate ICL of univariate linear functions. We experiment with a range of GPT-2-like transformer models trained from scratch. Our findings challenge the prevailing narrative that transformers adopt algorithmic approaches like linear regression to learn a linear function in-context. These models fail to generalize beyond their training distribution, highlighting fundamental limitations in their capacity to infer abstract task structures. Our experiments lead us to propose a mathematically precise hypothesis of what the model might be learning.

Problem

• Large Language Models
• Mathematical Relationships
• Abstract Understanding

Innovation

• Large Language Models
• Mathematical Concept Learning
• Generalization Limitations