From a Word-Level Dictionary to Sentence-Level Semantics: Multilingual Grievance Labelling with Contextual Models

2026-07-23

AI Summary

This study addresses the limitations of traditional dictionary-based approaches to grievance detection, which struggle with pragmatic phenomena such as assertion, quotation, negation, and condemnation in context, and are often evaluated under circular bias. Retaining the original 22-category grievance ontology, the authors replace lexical matching with a context-aware language model and introduce the first non-circular, multilingual benchmark dataset for grievance annotation, covering five languages. By modeling entire social media posts rather than isolated target sentences, the approach effectively captures implicit, cross-sentence, and quoted expressions of grievance. Experimental results demonstrate a substantial improvement in average precision—from 0.14 to 0.20—on texts not covered by existing dictionaries, significantly enhancing the detection of grievance signals in complex contextual settings.

Abstract

Grievance is one of the warning signs analysts look for when assessing threats of violence. It is increasingly measured at scale from online text, most often with word-level lexicons like the Grievance Dictionary that score by matching weighted terms. Such matching is a fast and transparent proxy, but it cannot resolve whether a term is asserted, quoted, negated, or condemned. These lexicons are also often evaluated on pools enriched with the very examples they retrieve, so a high score partly reflects agreement with the lexicon's own selection rule. Examining a five-language, 2{,}000-item evaluation pool, we find its halves separated almost perfectly by the lexicon itself: every item labeled ``random'' is in fact lexicon-negative, so the lexicon's apparent macro-AUROC of 0.686 collapses to a 0.500 floor fixed by construction. We keep the dictionary's 22-construct ontology but replace term matching with context-reading models, evaluated on a non-circular benchmark that separates unconditional-random, lexicon-positive, and lexicon-negative strata across five languages. Reading the full post rather than the target sentence alone helps most where the lexicon is silent, raising average precision on lexicon-negative text from 0.14 to 0.20, with the largest gains on quoted, implicit, and cross-sentence grievance. Together, these results show that grievance is measured more faithfully by reading the surrounding context, and more honestly when tested on text the lexicon did not select. We release our code and benchmark at https://github.com/behavioral-ds/multilingual_grievance.

Problem

• grievance
• lexicon-based measurement
• contextual semantics
• evaluation bias
• multilingual text analysis

Innovation

• contextual models
• multilingual grievance labeling
• non-circular evaluation
• sentence-level semantics
• lexicon-negative detection