R2T: Rule-Encoded Loss Functions for Low-Resource Sequence Tagging

2025-10-11

AI Summary

Low-resource sequence labeling (e.g., POS tagging, NER) faces challenges in handling out-of-vocabulary (OOV) words and severe scarcity of annotated data. Method: This paper proposes R2T—a rule-regularized training framework that encodes multi-level linguistic rules as adaptive regularization terms embedded directly into the loss function, enabling rule-guided neural training. Built upon a BiLSTM architecture, R2T integrates unsupervised learning with principled uncertainty modeling to autonomously induce OOV handling mechanisms from unlabeled data alone, establishing a novel “principled learning” paradigm. Contribution/Results: On Zarma POS tagging, R2T achieves 98.2% accuracy using only unlabeled data—surpassing AfriBERTa trained on 300 labeled sentences. For NER, fine-tuning on merely 50 annotated sentences exceeds the performance of baselines trained on 300 sentences. R2T pioneers joint rule–loss modeling, substantially reducing reliance on labeled data.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Unsupervised & Self-Supervised Learning
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

We introduce the Rule-to-Tag (R2T) framework, a hybrid approach that integrates a multi-tiered system of linguistic rules directly into a neural network's training objective. R2T's novelty lies in its adaptive loss function, which includes a regularization term that teaches the model to handle out-of-vocabulary (OOV) words with principled uncertainty. We frame this work as a case study in a paradigm we call principled learning (PrL), where models are trained with explicit task constraints rather than on labeled examples alone. Our experiments on Zarma part-of-speech (POS) tagging show that the R2T-BiLSTM model, trained only on unlabeled text, achieves 98.2% accuracy, outperforming baselines like AfriBERTa fine-tuned on 300 labeled sentences. We further show that for more complex tasks like named entity recognition (NER), R2T serves as a powerful pre-training step; a model pre-trained with R2T and fine-tuned on just 50 labeled sentences outperformes a baseline trained on 300.

Problem

• Integrates linguistic rules into neural network training objectives
• Handles out-of-vocabulary words with principled uncertainty mechanisms
• Enables low-resource sequence tagging using minimal labeled data

Innovation

• Integrates linguistic rules into neural network training
• Uses adaptive loss function for out-of-vocabulary words
• Enables learning from unlabeled text with task constraints