Zero-shot Dependency Parsing with Unsupervised Cross-Lingual Bootstrapping

2026-09-29

AI Summary

This study addresses the limited zero-shot cross-lingual transfer performance of pretrained language models in syntactic dependency parsing by proposing an unsupervised cross-lingual bootstrapping method. Built upon an encoder architecture, the approach strengthens the model's internal syntactic representations through a novel bootstrapping mechanism and introduces a parameter-free tree probing technique for validation. Experimental results demonstrate that the proposed method significantly improves zero-shot parsing performance for low-resource languages. Furthermore, probing evaluations confirm that the approach effectively enhances the syntactic robustness of the underlying model.

Abstract

Pre-trained language models (PLMs) with encoder-based architectures have shown impressive capabilities in zero-shot cross-lingual transfer for various language understanding tasks. However, applying this technique to dependency parsing remains a significant challenge due to its syntactic nature. To boost model generalizability across linguistic typologies, we propose a cross-lingual unsupervised bootstrapping method to improve syntactic knowledge within the PLM. We show that our method achieves a significant improvement in zero-shot parsing performance in low-resource languages. Analysis of these bootstrapped models uncovers increased robustness in recognizing syntactic structures, evidenced by higher scores in parameter-free tree probing tests.

Problem

• Zero-shot Dependency Parsing
• Cross-Lingual Transfer
• Pre-trained Language Models
• Low-resource Languages

Innovation

• Zero-shot Dependency Parsing
• Unsupervised Cross-Lingual Bootstrapping
• Pre-trained Language Models
• Syntactic Knowledge
• Tree Probing