part-of-speech tagging

This work addresses the problem of part-of-speech (POS) tagging for source code identifiers—a task where general-purpose NLP tools suffer from poor generalization in code contexts. To overcome this, we propose the first syntax-pattern mapping method customized to developers’ naming conventions. Our approach constructs a manually validated ground-truth repository linking identifiers to syntactic patterns and designs a feature engineering pipeline integrating character-level features with domain-specific knowledge. We employ a GradientBoostingClassifier for end-to-end mapping. The key contribution lies in formulating identifier naming as a natural-language syntax-pattern sequence generation task—enabled by domain-specific ground truth and tailored features—which substantially improves generalization. Experimental results demonstrate statistically significant gains in accuracy over conventional POS taggers and state-of-the-art code analysis tools. The implementation is publicly available.

Standard evaluation in NLP typically indicates that system A is better on average than system B, but it provides little info on how to improve performance and, what is worse, it should not come as a surprise if B ends up being better than A on outside data. We propose an evaluation methodology for sequence labeling tasks grounded on error analysis that provides both quantitative and qualitative information on where systems must be improved and predicts how models will perform on a different distribution. The key is to create test sets that, contrary to common practice, do not rely on gathering large amounts of real-world in-distribution scraped data, but consists in handcrafting a small set of linguistically motivated examples that exhaustively cover the range of span attributes (such as shape, length, casing, sentence position, etc.) a system may encounter in the wild. We demonstrate this methodology on a benchmark for anglicism identification in Spanish. Our methodology provides results that are diagnostic (because they help identify systematic weaknesses in performance), actionable (because they can inform which model is better suited for a given scenario) and predictive: our method predicts model performance on external datasets with a median correlation of 0.85.

To address unintended memorization of private or copyrighted content in large language model (LLM) pretraining data, this paper proposes KeyMIA, a keyword-driven membership inference attack. KeyMIA is the first method to incorporate semantic-aware keyword importance modeling into the membership inference framework. It operates in two stages: semantic-sensitive keyword tagging and fine-grained discrimination via average log-likelihood scoring (Tabbing), thereby enabling word-level data provenance—surpassing conventional token-level probability statistics. Technically, it integrates NLP keyword extraction, LLM probability output parsing, and statistical analysis. Evaluated on three benchmarks—BookMIA, MIMIR, and The Pile—KeyMIA achieves AUC improvements of 4.1–12.1% over state-of-the-art methods, significantly enhancing both interpretability and accuracy in detecting data leakage.

Manual annotation suffers from inconsistent quality and lacks systematic evaluation. Method: This paper proposes a consensus-based quality measurement framework grounded in multi-round annotation statistics, using dynamic decay of inter-annotator agreement variance as the core metric—established here as a “gold standard” for data quality. Recognizing annotators’ significant warm-up period but prohibitive cost of full-sample multiple annotation, we design a low-redundancy, high-efficiency progressive annotation protocol. The approach integrates statistical consistency analysis, variance convergence modeling, and label confidence estimation. Contribution/Results: Our paradigm substantially enhances data quality’s measurability and controllability: experiments show 3.2–7.8% accuracy gains across multiple NLP tasks and over 30% reduction in annotation redundancy.

Large language models lack explicit mechanisms to refer to specific spans in the input text, leading to inconsistent performance with existing span annotation prompting strategies. This work systematically examines three categories of approaches: input tagging, numerical indexing, and content matching, and proposes LogitMatch—a novel constrained decoding method that enforces alignment between model outputs and valid input spans in logit space to address the inconsistency inherent in content matching. Experiments across four diverse tasks demonstrate that LogitMatch significantly outperforms existing content matching methods and, in certain settings, surpasses other strategies, while also confirming that input tagging remains a robust baseline.

Low-resource sequence labeling (e.g., POS tagging, NER) faces challenges in handling out-of-vocabulary (OOV) words and severe scarcity of annotated data. Method: This paper proposes R2T—a rule-regularized training framework that encodes multi-level linguistic rules as adaptive regularization terms embedded directly into the loss function, enabling rule-guided neural training. Built upon a BiLSTM architecture, R2T integrates unsupervised learning with principled uncertainty modeling to autonomously induce OOV handling mechanisms from unlabeled data alone, establishing a novel “principled learning” paradigm. Contribution/Results: On Zarma POS tagging, R2T achieves 98.2% accuracy using only unlabeled data—surpassing AfriBERTa trained on 300 labeled sentences. For NER, fine-tuning on merely 50 annotated sentences exceeds the performance of baselines trained on 300 sentences. R2T pioneers joint rule–loss modeling, substantially reducing reliance on labeled data.

Language reasoning models (LRMs) suffer from low generation efficiency due to redundant verification and reflection steps. This paper proposes Step-Tagging, a lightweight framework that enables controllable intervention during inference by real-time identification and tagging of reasoning step types. Its core contributions are threefold: (1) the first principled ReasonType taxonomy for classifying reasoning steps; (2) an interpretable, step-count–based online early-stopping mechanism supporting dynamic termination; and (3) a lightweight sentence classifier integrated with a custom monitoring strategy. Evaluated on MATH500, GSM8K, AIME, GPQA, and MMLU-Pro, Step-Tagging achieves 20%–50% token reduction with zero accuracy degradation—yielding the greatest computational savings on highly demanding reasoning tasks.

This study addresses the UniDive 2025 morphosyntactic parsing shared task by proposing a joint multi-task framework that unifies modeling of morphological features, dependency syntactic structure, and content-word identification across nine languages. Methodologically, it employs XLM-RoBERTa as a shared encoder and couples three specialized decoders—one for each task—augmented with fine-grained error analysis and ablation studies. The key contributions are: (i) the first end-to-end joint prediction of morphology, syntax, and content words under the latest Universal Dependencies (UD) annotation scheme, significantly improving cross-lingual consistency; and (ii) empirical validation that content-word identification provides critical performance gains. Experiments show the model achieves average MS-LAS of 78.7%, LAS of 80.1%, and Feats F1 of 90.3% across all nine languages, outperforming prior systems in overall performance.

This work proposes a computational approach to automatically induce a broad-coverage, interpretable construction grammar from large-scale, jointly annotated syntactic and semantic corpora, modeling the complex mapping between syntactic structures and semantic relations. Leveraging the Fluid Construction Grammar framework, we present the first large-scale automatic induction of constructions, resulting in a network comprising tens of thousands of grammatical constructions. The system not only demonstrates the scalability of construction grammar theory under big-data conditions but also provides an efficient and interpretable tool for open-domain semantic parsing and the study of English argument structure. Furthermore, it reveals rich usage patterns linking syntax and semantics embedded in natural language data.

To address the low efficiency and high cost of manual grammatical annotation in corpus linguistics, this paper proposes an LLM-driven unsupervised grammatical annotation framework requiring no labeled data. Methodologically, we design a four-stage pipeline comprising prompt engineering, pre-evaluation filtering, batched API invocation, and pre-/post-annotation validation, leveraging GPT-5 as the foundational model to achieve end-to-end automated annotation on the Corpus of Historical American English (COHA). Our key contribution is the first systematic integration of large language models into large-scale diachronic grammatical analysis, balancing annotation accuracy with computational scalability. Experimental results demonstrate that the framework annotates 143,000 sentences within 60 hours at >98% accuracy, drastically reducing human intervention and temporal overhead. This work establishes a reproducible, robust paradigm for large-scale, evolution-oriented linguistic research grounded in historical corpora.