Multilingual Relative Clause Attachment Ambiguity Resolution in Large Language Models

2025-03-04

AI Summary

This study systematically evaluates large language models’ (LLMs) ability to resolve relative clause (RC) attachment ambiguity across six languages—English, Spanish, French, German, Japanese, and Korean—comparing their performance against human syntactic processing patterns. It specifically examines the effects of RC length and the syntactic position of complex determiner phrases (DPs). Using prompt engineering and controlled cloze tasks, we assess leading models including Claude, Gemini, and Llama. Results show that LLMs approximate human performance in Indo-European languages but exhibit significant degradation in Japanese and Korean, frequently misaligning with native syntactic structures—often “back-translating” into English-like parses—revealing a path dependency on English-centric training paradigms. To our knowledge, this is the first cross-linguistic, multi-model study focused explicitly on syntactic ambiguity resolution. It uncovers a fundamental limitation in LLMs’ non-European language modeling and provides critical empirical evidence and concrete directions for developing truly multilingual, robust syntactic competence.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Analogy

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This study examines how large language models (LLMs) resolve relative clause (RC) attachment ambiguities and compares their performance to human sentence processing. Focusing on two linguistic factors, namely the length of RCs and the syntactic position of complex determiner phrases (DPs), we assess whether LLMs can achieve human-like interpretations amid the complexities of language. In this study, we evaluated several LLMs, including Claude, Gemini and Llama, in multiple languages: English, Spanish, French, German, Japanese, and Korean. While these models performed well in Indo-European languages (English, Spanish, French, and German), they encountered difficulties in Asian languages (Japanese and Korean), often defaulting to incorrect English translations. The findings underscore the variability in LLMs' handling of linguistic ambiguities and highlight the need for model improvements, particularly for non-European languages. This research informs future enhancements in LLM design to improve accuracy and human-like processing in diverse linguistic environments.

Problem

• Resolving relative clause attachment ambiguities in multilingual contexts.
• Assessing LLM performance compared to human sentence processing.
• Identifying challenges in LLMs for non-European languages.

Innovation

• Evaluates LLMs on RC attachment ambiguities
• Assesses human-like interpretations in multiple languages
• Highlights need for improvements in non-European languages