Op-Fed: Opinion, Stance, and Monetary Policy Annotations on FOMC Transcripts Using Active Learning

2025-09-16

AI Summary

This work addresses two key challenges in identifying monetary policy stances in FOMC meeting minutes: severe class imbalance (non-neutral stance sentences constitute <8%) and heavy reliance on cross-sentence context (65% of instances). To this end, we introduce Op-Fed—a high-quality, context-enhanced dataset comprising 1,044 human-annotated sentences with their surrounding discourse. We propose a five-stage hierarchical annotation framework that decouples opinion, policy, and stance components, integrated with active learning to double the number of rare-class positive samples. Our context-aware annotation schema significantly improves feasibility for fine-grained stance classification. Experiments show that state-of-the-art closed-source LLMs achieve zero-shot accuracy of 0.80 on opinion classification but only 0.61 on stance classification—substantially below the human baseline of 0.89—demonstrating Op-Fed’s rigor and benchmark value. This work establishes the first structured, context-rich, domain-specific resource and methodological paradigm for stance analysis in financial texts.

Technology Category

• Natural Language Processing: Sentiment Analysis, Stylistic Analysis, and Argument Mining
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Web Mining and Content Analysis: Sentiment analysis and opinion mining
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The U.S. Federal Open Market Committee (FOMC) regularly discusses and sets monetary policy, affecting the borrowing and spending decisions of millions of people. In this work, we release Op-Fed, a dataset of 1044 human-annotated sentences and their contexts from FOMC transcripts. We faced two major technical challenges in dataset creation: imbalanced classes -- we estimate fewer than 8% of sentences express a non-neutral stance towards monetary policy -- and inter-sentence dependence -- 65% of instances require context beyond the sentence-level. To address these challenges, we developed a five-stage hierarchical schema to isolate aspects of opinion, monetary policy, and stance towards monetary policy as well as the level of context needed. Second, we selected instances to annotate using active learning, roughly doubling the number of positive instances across all schema aspects. Using Op-Fed, we found a top-performing, closed-weight LLM achieves 0.80 zero-shot accuracy in opinion classification but only 0.61 zero-shot accuracy classifying stance towards monetary policy -- below our human baseline of 0.89. We expect Op-Fed to be useful for future model training, confidence calibration, and as a seed dataset for future annotation efforts.

Problem

• Annotating FOMC transcripts for opinion and monetary policy stance
• Addressing class imbalance and inter-sentence context dependence
• Evaluating LLM performance on monetary policy classification tasks

Innovation

• Active learning for imbalanced dataset annotation
• Hierarchical schema for opinion and stance classification
• Context-aware annotation to address inter-sentence dependence