DecIF: Improving Instruction-Following through Meta-Decomposition

2025-05-20

AI Summary

Existing instruction-tuning methods heavily rely on external documents or human annotations to construct high-quality instruction data, limiting their generalizability and scalability. To address this, we propose DecIF, a novel meta-decomposition-guided framework for fully automated instruction data synthesis. DecIF enables large language models to autonomously perform structured instruction generation, instruction–response consistency verification, and response validation and filtering based on atomic-level evaluation criteria—entirely without external resources or human intervention. By integrating meta-information-driven iterative generation with self-supervised response optimization, DecIF achieves significant improvements over state-of-the-art baselines across diverse instruction-following tasks. Experimental results demonstrate superior flexibility, generalization capability, and deployment efficiency, establishing a new paradigm for resource-efficient, scalable instruction tuning.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Instruction-following has emerged as a crucial capability for large language models (LLMs). However, existing approaches often rely on pre-existing documents or external resources to synthesize instruction-following data, which limits their flexibility and generalizability. In this paper, we introduce DecIF, a fully autonomous, meta-decomposition guided framework that generates diverse and high-quality instruction-following data using only LLMs. DecIF is grounded in the principle of decomposition. For instruction generation, we guide LLMs to iteratively produce various types of meta-information, which are then combined with response constraints to form well-structured and semantically rich instructions. We further utilize LLMs to detect and resolve potential inconsistencies within the generated instructions. Regarding response generation, we decompose each instruction into atomic-level evaluation criteria, enabling rigorous validation and the elimination of inaccurate instruction-response pairs. Extensive experiments across a wide range of scenarios and settings demonstrate DecIF's superior performance on instruction-following tasks. Further analysis highlights its strong flexibility, scalability, and generalizability in automatically synthesizing high-quality instruction data.

Problem

• Enhancing LLM instruction-following without external data reliance
• Generating diverse high-quality instructions via meta-decomposition
• Ensuring instruction-response consistency through atomic-level validation

Innovation

• Autonomous meta-decomposition framework for data generation
• Iterative meta-information synthesis for rich instructions
• Atomic-level criteria for rigorous response validation