Problem
Research questions and friction points this paper is trying to address.
Improves LLMs for complex programming tasks
Enhances modularity and correctness of code solutions
Introduces Modular-of-Thought framework for task decomposition
MoTCoder: Elevating Large Language Models with Modular of Thought for Challenging Programming Tasks
🤖 AI Summary
Large language models (LLMs) exhibit degraded correctness on complex programming tasks due to their tendency to generate monolithic, unstructured code.
Method: This paper proposes Modular Thinking (MoT), a framework that guides LLMs to decompose problems into logical subtasks and generate reusable, structured submodules. We introduce a novel MoT instruction-tuning paradigm that jointly optimizes task decomposition and modular code generation, incorporating structure-aware training objectives.
Contribution/Results: Evaluated on APPS and CodeContests benchmarks, MoT achieves absolute pass@1 improvements of 12.9% and 9.43%, respectively. It significantly enhances solution structure, maintainability, and reasoning correctness. To our knowledge, this is the first work to explicitly integrate modular modeling into the fine-tuning pipeline of programming-focused LLMs, establishing a new paradigm for synthesizing complex programs through compositional, interpretable modules.
📝 Abstract
Large Language Models (LLMs) have showcased impressive capabilities in handling straightforward programming tasks. However, their performance tends to falter when confronted with more challenging programming problems. We observe that conventional models often generate solutions as monolithic code blocks, restricting their effectiveness in tackling intricate questions. To overcome this limitation, we present Modular-of-Thought Coder (MoTCoder). We introduce a pioneering framework for MoT instruction tuning, designed to promote the decomposition of tasks into logical sub-tasks and sub-modules. Our investigations reveal that, through the cultivation and utilization of sub-modules, MoTCoder significantly improves both the modularity and correctness of the generated solutions, leading to substantial relative pass@1 improvements of 12.9% on APPS and 9.43% on CodeContests. Our codes are available at https://github.com/dvlab-research/MoTCoder.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Modular-of-Thought Coder (MoTCoder) framework introduced
Decomposes tasks into logical sub-tasks and sub-modules
Improves modularity and correctness of generated solutions
🔎 Similar Papers
No similar papers found.
