SAGA: Score-Weighted Adaptive Generation Alignment for Low-Resource Nordic Language Models

2026-08-06

AI Summary

This work addresses the challenge of improving grammatical quality in low-resource Nordic languages—Danish, Icelandic, and Norwegian Bokmål—where the scarcity of human-annotated preference data hinders conventional alignment methods. The authors propose SAGA, a novel framework that leverages dependency parser outputs to automatically generate syntax-guided preference pairs without manual annotation. By integrating parsing accuracy and lexical diversity into a composite reward signal, and further enhancing training stability through reward margin filtering and cheating detection mechanisms, SAGA effectively aligns language models with grammatical norms. Experimental results demonstrate substantial gains: Danish parsing success rates rise from 69.0% to 93.8%, Icelandic models show an average 3.3-point improvement in Stanza-based scores, native speakers prefer SAGA-generated outputs in 80% of pairwise comparisons, and Norwegian performance increases by 28 percentage points.

Abstract

Preference optimisation has proven effective for improving large language models but typically relies on costly human preference annotations. Extending these methods to morphologically rich, low-resource languages remains challenging because such annotations are scarce. We present SAGA (Score-weighted Adaptive Generation Alignment), a parser-guided preference optimisation framework that replaces human labels with dependency-parser supervision. SAGA converts parser judgements into preference pairs for delta-DPO, combines parser quality with lexical diversity in a composite reward, filters low-information pairs using a reward-gap criterion, and monitors reward hacking to maintain reliable supervision. Across Danish, Icelandic, and Norwegian Bokmål using GPT-SW3-1.3B, SAGA consistently improves grammatical quality without requiring human preference labels. Danish parse success increases from 69.0% to 93.8%, Icelandic achieves a +4.5 percentage-point improvement on an independent Stanza evaluation (three-run mean +3.3 percentage points) while native speakers prefer SAGA outputs in 80% of pairwise comparisons, and Norwegian Bokmål improves by +28 percentage points. These results demonstrate that parser-derived supervision is a practical alternative to human preference annotation for grammatical alignment in low-resource languages where high-quality dependency parsers are available.

Problem

• preference optimisation
• low-resource languages
• human preference annotations
• grammatical alignment
• dependency parsing

Innovation

• preference optimisation
• dependency parsing
• low-resource languages
• delta-DPO
• reward hacking