Character-aware Transformers Learn an Irregular Morphological Pattern Yet None Generalize Like Humans

2026-02-15

AI Summary

This study investigates whether neural networks can acquire and generalize irregular morphological patterns—such as Spanish L-stem verbs—that lack phonological, semantic, or syntactic motivation, in a manner comparable to human learners. Using character-level encoder–decoder Transformers, the research systematically compares the effects of sequential versus position-invariant positional encodings and atomic versus decomposed morphological tag representations on learning outcomes. The work reveals, for the first time, the critical role of position-invariant encoding in capturing abstract morphological categories. Although position-invariant models successfully reproduce L-stem paradigm clustering, all models fail to exhibit the human-like preference for generalizing novel verbs to the first-person singular indicative form; instead, they only achieve mood-based generalization. This discrepancy indicates a fundamental divergence between current neural architectures and human cognitive mechanisms in morphological generalization.

Technology Category

• Natural Language Processing: Lexical Semantics and Morphology
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Social networks and social learning
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Whether neural networks can serve as cognitive models of morphological learning remains an open question. Recent work has shown that encoder-decoder models can acquire irregular patterns, but evidence that they generalize these patterns like humans is mixed. We investigate this using the Spanish \emph{L-shaped morphome}, where only the first-person singular indicative (e.g., \textit{pongo} `I put') shares its stem with all subjunctive forms (e.g., \textit{ponga, pongas}) despite lacking apparent phonological, semantic, or syntactic motivation. We compare five encoder-decoder transformers varying along two dimensions: sequential vs. position-invariant positional encoding, and atomic vs. decomposed tag representations. Positional encoding proves decisive: position-invariant models recover the correct L-shaped paradigm clustering even when L-shaped verbs are scarce in training, whereas sequential positional encoding models only partially capture the pattern. Yet none of the models productively generalize this pattern to novel forms. Position-invariant models generalize the L-shaped stem across subjunctive cells but fail to extend it to the first-person singular indicative, producing a mood-based generalization rather than the L-shaped morphomic pattern. Humans do the opposite, generalizing preferentially to the first-person singular indicative over subjunctive forms. None of the models reproduce the human pattern, highlighting the gap between statistical pattern reproduction and morphological abstraction.

Problem

• morphological learning
• irregular morphology
• neural generalization
• L-shaped morphome
• cognitive modeling

Innovation

• position-invariant positional encoding
• morphological generalization
• L-shaped morphome
• character-aware Transformers
• encoder-decoder models