Dependency Parsing with the Structuralized Prompt Template

2025-02-24

AI Summary

Dependency parsing aims to model syntactic dependency relations among words in a sentence. This paper proposes a purely prompt-driven, text-to-text dependency parsing approach that leverages only pretrained sequence-to-sequence encoders (e.g., T5 or BART), eliminating task-specific decoders and parameters. Our method introduces three key innovations: (1) a structured prompt template that explicitly encodes the topological constraints of dependency trees; (2) a linearized textual serialization of dependency structures; and (3) high-accuracy parsing without any parameter updates—i.e., zero-parameter fine-tuning. Experiments across multilingual benchmarks demonstrate that our approach matches or surpasses state-of-the-art parsers in accuracy, while exhibiting strong cross-model and cross-lingual plug-and-play capability and zero-shot transfer performance. To our knowledge, this is the first work to empirically validate the effectiveness and generalizability of the pure prompting paradigm for syntactic parsing.

Technology Category

• Natural Language Processing: Syntax — Tagging, Chunking & Parsing
• Planning, Routing, and Scheduling: Planning with Language Models
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Dependency parsing is a fundamental task in natural language processing (NLP), aiming to identify syntactic dependencies and construct a syntactic tree for a given sentence. Traditional dependency parsing models typically construct embeddings and utilize additional layers for prediction. We propose a novel dependency parsing method that relies solely on an encoder model with a text-to-text training approach. To facilitate this, we introduce a structured prompt template that effectively captures the structural information of dependency trees. Our experimental results demonstrate that the proposed method achieves outstanding performance compared to traditional models, despite relying solely on a pre-trained model. Furthermore, this method is highly adaptable to various pre-trained models across different target languages and training environments, allowing easy integration of task-specific features.

Problem

• Develops a novel dependency parsing method
• Uses structured prompt template for syntactic trees
• Enhances adaptability across languages and models

Innovation

• Text-to-text training approach
• Structured prompt template
• Pre-trained model integration