Filling the Gap: Is Commonsense Knowledge Generation useful for Natural Language Inference?

2025-07-20

AI Summary

Existing commonsense knowledge resources are insufficiently comprehensive and难以 support diverse premise-hypothesis pairs in Natural Language Inference (NLI). Method: This paper proposes dynamically generating commonsense knowledge using large language models (LLMs) and systematically evaluating its effectiveness in NLI. We design a dual-dimensional evaluation metric—factuality and consistency—to quantify the reliability of generated knowledge, and introduce a knowledge integration mechanism to inject it into mainstream NLI models. Contribution/Results: Explicit incorporation of LLM-generated commonsense knowledge does not uniformly improve overall accuracy but significantly enhances entailment recognition (+2.1%) and marginally improves contradiction and neutral classification. Empirical analysis reveals differential reliance on commonsense across inference types, confirming the type-sensitivity of commonsense augmentation. Our work establishes an evaluable, reproducible paradigm for knowledge generation and integration in commonsense-driven NLI.

Technology Category

Knowledge Representation and Reasoning: Common-Sense ReasoningNatural Language Processing: GenerationMachine Learning: Large Multimodal Models (LMMs)

Application Category

Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsUser Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendationSearch and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Natural Language Inference (NLI) is the task of determining the semantic entailment of a premise for a given hypothesis. The task aims to develop systems that emulate natural human inferential processes where commonsense knowledge plays a major role. However, existing commonsense resources lack sufficient coverage for a variety of premise-hypothesis pairs. This study explores the potential of Large Language Models as commonsense knowledge generators for NLI along two key dimensions: their reliability in generating such knowledge and the impact of that knowledge on prediction accuracy. We adapt and modify existing metrics to assess LLM factuality and consistency in generating in this context. While explicitly incorporating commonsense knowledge does not consistently improve overall results, it effectively helps distinguish entailing instances and moderately improves distinguishing contradictory and neutral inferences.

Problem

• Assessing LLM reliability in generating commonsense knowledge for NLI
• Evaluating impact of generated knowledge on NLI prediction accuracy
• Addressing coverage gaps in existing commonsense resources for NLI

Innovation

• Using LLMs as commonsense knowledge generators
• Adapting metrics for LLM factuality assessment
• Enhancing NLI with generated commonsense knowledge