Abductive Logical Rule Induction by Bridging Inductive Logic Programming and Multimodal Large Language Models

2025-09-26

AI Summary

This work addresses the challenge of jointly discovering logical facts and inducing logical rules from unstructured text and images under few-shot settings. We propose ILP-CoT, the first framework that synergistically integrates the rule-generation capability of multimodal large language models (MLLMs) with the formal reasoning of inductive logic programming (ILP). ILP-CoT operates via bidirectional collaboration: MLLMs generate structured candidate rules to constrain the ILP search space, while ILP formally verifies and refines these candidates—thereby mitigating MLLM hallucinations and reducing ILP’s computational overhead. Evaluated on multiple logical reasoning benchmarks, ILP-CoT significantly outperforms both pure ILP and pure MLLM baselines. Furthermore, it successfully transfers to rule-driven text-to-image generation, demonstrating high reliability, strong interpretability, and cross-task generalization capability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Multi-modal Vision

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

We propose ILP-CoT, a method that bridges Inductive Logic Programming (ILP) and Multimodal Large Language Models (MLLMs) for abductive logical rule induction. The task involves both discovering logical facts and inducing logical rules from a small number of unstructured textual or visual inputs, which still remain challenging when solely relying on ILP, due to the requirement of specified background knowledge and high computational cost, or MLLMs, due to the appearance of perceptual hallucinations. Based on the key observation that MLLMs could propose structure-correct rules even under hallucinations, our approach automatically builds ILP tasks with pruned search spaces based on the rule structure proposals from MLLMs, and utilizes ILP system to output rules built upon rectified logical facts and formal inductive reasoning. Its effectiveness is verified through challenging logical induction benchmarks, as well as a potential application of our approach, namely text-to-image customized generation with rule induction. Our code and data are released at https://github.com/future-item/ILP-CoT.

Problem

• Bridging ILP and MLLMs for abductive logical rule induction
• Discovering logical facts from unstructured textual or visual inputs
• Overcoming limitations of ILP background knowledge and MLLM hallucinations

Innovation

• Bridges ILP and MLLMs for logical rule induction
• Automatically builds ILP tasks with pruned search spaces
• Utilizes ILP system for formal inductive reasoning