Frequency matters: Modeling irregular morphological patterns in Spanish with Transformers

2024-10-28

arXiv.org

AI Summary

This study investigates morphological reinflection of Spanish L-type irregular verbs, examining how input frequency influences Transformer models’ acquisition of regular versus L-type irregular patterns. Using the paradigm completion (PCFP) framework, we employ controlled data distribution manipulation, behavioral analysis, and error diagnostics to systematically uncover model behavior under frequency imbalance. Key contributions: (1) Models achieve significantly higher accuracy on L-type verbs than on regular verbs—evidencing an “irregularity preference”; (2) Increasing L-type verb proportion strengthens memory-dominated processing, supporting a hybrid analogy–memory account; (3) Sparse consonantal alternations trigger systematic regularization errors, revealing limitations in modeling rare phonological constraints. We provide the first quantitative confirmation of both primacy effects and frequency-dependent memory enhancement in morphological learning. These findings offer novel empirical evidence for cognitive-computational models of morphological acquisition and advance our understanding of how neural architectures balance rule application and lexical memory in inflectional morphology.

Technology Category

• Natural Language Processing: Lexical Semantics and Morphology
• Cognitive Modeling & Cognitive Systems: Analogy
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Models for Web evolution
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Over the past decade, various studies have addressed how speakers solve the so-called `The Paradigm Cell Filling Problem' (PCFP) citep{ackerman2009parts} across different languages. The PCFP addresses a fundamental question in morphological processing: how do speakers accurately generate inflected forms of words when presented with incomplete paradigms? This problem is particularly salient when modeling complex inflectional systems. We focus on Spanish verbal paradigms, where certain verbs follow an irregular L-shaped pattern, where the first-person singular present indicative stem matches the stem used throughout the present subjunctive mood. We formulate the problem as a morphological reinflection task. Specifically, we investigate the role of input frequency in the acquisition of regular versus irregular L-shaped patterns in transformer models. By systematically manipulating the input distributions and analyzing model behavior, we reveal four key findings: 1) Models perform better on L-shaped verbs compared to regular verbs, especially in uneven frequency conditions; 2) Robust primacy effects are observed, but no consistent recency effects; 3) Memorization becomes more prominent as the proportion of L-shaped verbs increases; 4) There is a tendency to regularize L-shaped verbs when their consonant alternation pairs are rare or absent in the training data.

Problem

• Model irregular Spanish verb morphology using Transformers
• Examine frequency impact on regular vs. irregular patterns
• Analyze training data effects on morphological reinflection accuracy

Innovation

• Transformers model Spanish morphology
• Focus on irregular L-shaped patterns
• Analyze input frequency impact