Investigating Language Model Capabilities to Represent and Process Formal Knowledge: A Preliminary Study to Assist Ontology Engineering

2025-09-12

AI Summary

Large language models (LMs) exhibit weak reasoning capabilities and poor controllability in ontology engineering tasks. Method: This work investigates the feasibility of enhancing formal knowledge representation and reasoning performance in small language models (SLMs) by replacing natural language inputs with compact formal logic syntax—specifically, fragments of description logic—and conducts controlled experiments to systematically evaluate how syntactic formalization affects SLM performance on core ontology tasks, including consistency checking and classification. Contribution/Results: Formalized inputs significantly improve model interpretability and controllability while maintaining or even surpassing natural-language-based accuracy across multiple reasoning tasks. These findings establish a novel paradigm for trustworthy ontology construction that synergistically integrates symbolic logic with neural models, providing empirical validation for neuro-symbolic integration in knowledge engineering.

Technology Category

Knowledge Representation and Reasoning: Description LogicsConstraint Satisfaction and Optimization: Satisfiability Modulo TheoriesMachine Learning: Neuro-Symbolic Learning

Application Category

Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsGraph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recent advances in Language Models (LMs) have failed to mask their shortcomings particularly in the domain of reasoning. This limitation impacts several tasks, most notably those involving ontology engineering. As part of a PhD research, we investigate the consequences of incorporating formal methods on the performance of Small Language Models (SLMs) on reasoning tasks. Specifically, we aim to orient our work toward using SLMs to bootstrap ontology construction and set up a series of preliminary experiments to determine the impact of expressing logical problems with different grammars on the performance of SLMs on a predefined reasoning task. Our findings show that it is possible to substitute Natural Language (NL) with a more compact logical language while maintaining a strong performance on reasoning tasks and hope to use these results to further refine the role of SLMs in ontology engineering.

Problem

• Assessing language models' formal knowledge representation capabilities
• Investigating compact logical languages for reasoning tasks
• Exploring small language models for ontology engineering assistance

Innovation

• Using logical language instead of natural language
• Incorporating formal methods into small language models
• Substituting NL with compact logical grammar