🤖 AI Summary
This study addresses the challenges of extracting structured risk factors from corporate 10-K filings, particularly inconsistencies with predefined hierarchical taxonomies and the absence of continuous optimization mechanisms. The authors propose an end-to-end, three-stage framework: first, a large language model (LLM) extracts risk factors with source citations; second, semantic embeddings map these factors to a taxonomy; and third, an LLM-as-a-judge mechanism filters erroneous matches. Additionally, an AI agent is introduced to autonomously diagnose and iteratively refine the taxonomy. Experiments on S&P 500 company filings demonstrate a 63% increase in within-industry risk similarity (Cohen’s d = 1.06, AUC = 0.82) and a 104.7% improvement in embedding separation between categories, confirming the method’s effectiveness and generalizability.
📝 Abstract
We present a methodology for extracting structured risk factors from corporate 10-K filings while maintaining adherence to a predefined hierarchical taxonomy. Our three-stage pipeline combines LLM extraction with supporting quotes, embedding-based semantic mapping to taxonomy categories, and LLM-as-a-judge validation that filters spurious assignments. To evaluate our approach, we extract 10,688 risk factors from S&P 500 companies and examine risk profile similarity across industry clusters. Beyond extraction, we introduce autonomous taxonomy maintenance where an AI agent analyzes evaluation feedback to identify problematic categories, diagnose failure patterns, and propose refinements, achieving 104.7% improvement in embedding separation in a case study. External validation confirms the taxonomy captures economically meaningful structure: same-industry companies exhibit 63% higher risk profile similarity than cross-industry pairs (Cohen's d=1.06, AUC 0.82, p<0.001). The methodology generalizes to any domain requiring taxonomy-aligned extraction from unstructured text, with autonomous improvement enabling continuous quality maintenance and enhancement as systems process more documents.