Brevity is the Soul of Wit: Condensing Code Changes to Improve Commit Message Generation

2025-09-19

AI Summary

Developers often neglect writing high-quality commit messages, hindering code comprehension and maintenance. To address limitations in existing automated approaches—particularly in change representation and organization—this paper proposes a three-part structured textual template comprising a change summary, comment prompts, and key identifiers. We introduce ChangeScribe to generate concise, semantically accurate change summaries and employ heuristic rules to compress code changes into the template. Furthermore, we fine-tune CodeLlama-7B to effectively process this structured input. Our method preserves human readability while substantially unlocking the capabilities of large language models. Evaluated on standard benchmarks, it outperforms six state-of-the-art baselines across all metrics: BLEU-Norm improves by 51.7%, METEOR by 78.7%, and ROUGE-L by 62.5%. These results empirically validate the efficacy of structured compression for commit message generation.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Commit messages are valuable resources for describing why code changes are committed to repositories in version control systems (e.g., Git). They effectively help developers understand code changes and better perform software maintenance tasks. Unfortunately, developers often neglect to write high-quality commit messages in practice. Therefore, a growing body of work is proposed to generate commit messages automatically. These works all demonstrated that how to organize and represent code changes is vital in generating good commit messages, including the use of fine-grained graphs or embeddings to better represent code changes. In this study, we choose an alternative way to condense code changes before generation, i.e., proposing brief yet concise text templates consisting of the following three parts: (1) summarized code changes, (2) elicited comments, and (3) emphasized code identifiers. Specifically, we first condense code changes by using our proposed templates with the help of a heuristic-based tool named ChangeScribe, and then fine-tune CodeLlama-7B on the pairs of our proposed templates and corresponding commit messages. Our proposed templates better utilize pre-trained language models, while being naturally brief and readable to complement generated commit messages for developers. Our evaluation based on a widely used dataset showed that our approach can outperform six baselines in terms of BLEU-Norm, METEOR, and ROUGE-L, with average improvements of 51.7%, 78.7%, and 62.5%, respectively. The ablation study and human evaluation also provide further insights into the effectiveness of our approach.

Problem

• Automating commit message generation for code changes
• Condensing code changes into concise text templates
• Improving readability and quality of generated commit messages

Innovation

• Condense code changes using text templates
• Fine-tune CodeLlama-7B on template-message pairs
• Utilize ChangeScribe for heuristic-based code summarization