Problem
Research questions and friction points this paper is trying to address.
Neuro-Symbolic predictors
symbolic constraints
auto-formalization
domain knowledge
logical formulas
Auto-Formalizing Neuro-Symbolic Predictors
🤖 AI Summary
This study addresses the high cost and heavy reliance on human expertise associated with formalizing domain knowledge in neuro-symbolic models. To mitigate this, we propose leveraging large language models (LLMs) to automatically translate textual knowledge into symbolic constraints. We introduce auto-nesy-bench, an evaluation benchmark designed to systematically assess the logical formula generation capabilities of LLMs and their impact on downstream prediction accuracy. Experimental results demonstrate that LLMs can autonomously generate valid logical formulas approaching expert-level quality, significantly enhancing the predictive performance of neuro-symbolic models. This research validates the feasibility of LLM-driven automated neuro-symbolic reasoning, offering a new paradigm for reducing the cost of domain knowledge injection.
📝 Abstract
Neuro-Symbolic (NeSy) predictors incorporate prior knowledge into the prediction process of neural networks, ensuring that outputs satisfy specified constraints, making them particularly suitable for high-stakes applications where compliance with domain knowledge is essential. A key bottleneck in this paradigm is the acquisition of symbolic constraints: encoding domain knowledge into logical formulas remains a manual and expert-intensive process. In this work, we investigate the extent to which auto-formalization via LLMs can systematically translate textual knowledge into symbolic knowledge that can be plugged into NeSy predictors. To this end, we introduce auto-nesy-bench, a new benchmark for evaluating constraint formalization and its impact on downstream accuracy of NeSy predictors. Through an extensive evaluation across several domains, we find that LLMs can formalize constraints to a meaningful extent, generating formulas that are often similar to those provided by human experts. Moreover, when the generated formulas are syntactically valid, they can lead to high-quality downstream predictions. The code and benchmark are available at https://unitn-sml.github.io/auto-nesy-bench/.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Auto-Formalization
Neuro-Symbolic Predictors
Large Language Models
Benchmark
Symbolic Constraints
🔎 Similar Papers
No similar papers found.
