Interpretable Column Annotation with LLM-Symbolized Decision Process Materialization

2026-07-27

AI Summary

Existing table column annotation methods are limited by their neglect of label semantics and insufficient interpretability and adaptability. This work proposes SymCA, a novel framework that formulates annotation as a symbolic decision process progressing from global to local levels. SymCA first leverages a large language model (LLM) to construct a tree-structured semantic skeleton and then employs executable and evolvable prediction primitives to enable node-level adaptive decisions. The approach integrates a minimum Bayes risk (MBR) consensus strategy with an exploration–exploitation-driven operator evolution mechanism and utilizes an interpretable random forest classifier. Experimental results demonstrate that SymCA substantially outperforms the strongest baseline, achieving average improvements of 6.42% and 11.03% in Micro-F1 and Macro-F1 scores, respectively, thereby significantly enhancing accuracy, robustness, and interpretability.

Abstract

Column annotation (CA), including column type annotation (CTA) and column property annotation (CPA), aims to identify the meanings of table columns and the semantic relationships among them. Recent CA methods usually use various neural models to learn column representations and directly map them to label categories, thereby (1) sacrificing model interpretability and adaptivity, and (2) overlooking rich label semantics and ultimately limiting accuracy. To address these limitations, we propose SymCA, an LLM-empowered interpretable CA framework that materializes column annotation as a global-to-local symbolic decision process. SymCA consists of two components: (1) global skeleton induction, which constructs a semantic skeleton over the label space, and (2) local substrate evolution, which evolves predictive substrates within the skeleton. Specifically, to exploit label semantics while preserving an interpretable decision process, the global skeleton induction module leverages LLMs to generate candidate hypernym-inspired tree-structured semantic skeletons and employs a Minimum Bayes Risk (MBR)-based consensus strategy to select a robust skeleton against generation variance. Since different internal nodes require different evidence to distinguish among their child nodes, the local substrate evolution module materializes each internal node as an executable and evolvable predictive substrate. Over multiple evolution rounds, each substrate trains an interpretable random forest classifier with the current operator set, leverages the LLM to propose node-specific operator modifications, and uses an exploration-exploitation strategy to prioritize promising substrates. Extensive experiments demonstrate that SymCA is accurate, robust, and interpretable, outperforming the strongest baselines by an average of 6.42% in Micro-F1 and 11.03% in Macro-F1.

Problem

• column annotation
• interpretability
• label semantics
• neural models
• semantic relationships

Innovation

• symbolic decision process
• interpretable column annotation
• LLM-enhanced reasoning
• semantic skeleton
• predictive substrate evolution