🤖 AI Summary
Prior automated code documentation generation research primarily targets code summarization, lacking context-aware approaches tailored to templated documentation (e.g., Javadoc) and suffering from the absence of high-quality, modern Java– and mainstream-framework–inclusive datasets. Method: We introduce the first context-aware Javadoc generation dataset, explicitly incorporating class/method signatures, call-site context, framework-specific APIs, and structured semantic information. Leveraging this dataset, we systematically evaluate open-source LLMs—including LLaMA-3.1, Gemma-2, Phi-3, Mistral, and Qwen-2.5—under zero-shot, few-shot, and fine-tuning paradigms. Contribution/Results: Experiments demonstrate that LLaMA-3.1 achieves consistently superior and robust performance across all settings, empirically validating the critical role of contextual modeling in templated documentation generation. Our dataset establishes a reproducible foundation for industrial-grade intelligent documentation systems, and our evaluation framework provides a practical technical pathway for future research and deployment.
📝 Abstract
Code documentation is essential to improve software maintainability and comprehension. The tedious nature of manual code documentation has led to much research on automated documentation generation. Existing automated approaches primarily focused on code summarization, leaving a gap in template-based documentation generation (e.g., Javadoc), particularly with publicly available Large Language Models (LLMs). Furthermore, progress in this area has been hindered by the lack of a Javadoc-specific dataset that incorporates modern language features, provides broad framework/library coverage, and includes necessary contextual information. This study aims to address these gaps by developing a tailored dataset and assessing the capabilities of publicly available LLMs for context-aware, template-based Javadoc generation. In this work, we present a novel, context-aware dataset for Javadoc generation that includes critical structural and semantic information from modern Java codebases. We evaluate five open-source LLMs (including LLaMA-3.1, Gemma-2, Phi-3, Mistral, Qwen-2.5) using zero-shot, few-shot, and fine-tuned setups and provide a comparative analysis of their performance. Our results demonstrate that LLaMA 3.1 performs consistently well and is a reliable candidate for practical, automated Javadoc generation, offering a viable alternative to proprietary systems.