CoCoRerank: Towards Conventional Commit Message Generation by Component and Candidate Consistency Reranking

2026-09-25

AI Summary

This study addresses the challenges of format compliance and structural deficiency in automated commit message generation by constructing a high-quality benchmark dataset and proposing a reranking framework based on component and candidate consistency. The method introduces a novel two-dimensional consistency reranking mechanism that leverages large language models for structured generation, integrating horizontal and vertical consensus across code changes, types, scopes, and subjects to achieve multi-dimensional alignment. Experimental results demonstrate that this framework significantly improves the prediction accuracy of structural components and the quality of subject generation, establishing an effective foundation for the automated generation of standardized commit messages.

Abstract

Commit messages are essential for understanding software changes, yet automatic commit message generation typically treats a message as an unstructured text sequence. This limits its ability to support standardized development workflows, where commit messages are often expected to follow the Conventional Commits Specification (CCS) in the form type (scope): subject. In this paper, we study conventional commit message generation under the complete CCS format. We construct a new benchmark of 86,688 high-quality commits collected from open-source GitHub repositories, with each message normalized into type, scope, and subject through structural normalization and semantic quality filtering. Based on this benchmark, we propose a two-dimensional consistency-based reranking framework named CoCoRerank for LLM-based generation. CoCoRerank exploits horizontal consistency among the code change, type, scope, and subject, as well as vertical consensus across multiple generated candidates. Experiments with representative CMG baselines, LLM generators, reranking strategies, and ablation variants show that CoCoRerank improves both structural component prediction and subject generation quality. The results demonstrate that complete CCS supervision and multidimensional consistency modeling provide an effective foundation for accurate and standardized commit message generation. The artifact is publicly released at https://github.com/bluewhalebug/CoCoRerank.

Problem

• Commit Message Generation
• Conventional Commits Specification
• Structured Text Generation
• Software Engineering

Innovation

• Conventional Commit Message Generation
• Consistency Reranking
• Large Language Models
• Multidimensional Consistency Modeling
• Benchmark Construction