How Do Users Revise Architectural Related Questions on Stack Overflow: An Empirical Study

2024-06-27

arXiv.org

AI Summary

This study presents the first systematic investigation of revision behaviors for Architecture-Related Questions (ARQs) on Stack Overflow, aiming to understand how users dynamically refine architecturally significant information. Based on a manually annotated dataset of 4,114 ARQs with revision histories, we employ qualitative coding, empirical analysis, and statistical testing. We find that ARQ revisions are infrequent yet highly time-sensitive—most occur before the first answer and are predominantly performed by askers. Fourteen categories of supplementary information are identified, with design context, component dependencies, and architectural concerns being the most prevalent. Primary revision motivations center on “clarifying architectural understanding” and “improving question readability.” Empirical evaluation demonstrates that revisions significantly enhance answer practicality, informativeness, and relevance. Our findings provide novel evidence for community-driven mechanisms underlying architectural knowledge evolution in Q&A platforms.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Knowledge Representation and Reasoning: Qualitative Reasoning
• Data Mining & Knowledge Management: Intelligent Query Processing

Application Category

• Web Mining and Content Analysis: Community question answering
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Technical Questions and Answers (Q&A) sites, such as Stack Overflow (SO), accumulate a significant variety of information related to software development in posts from users. To ensure the quality of this information, SO encourages its users to review posts through various mechanisms (e.g., question and answer revision processes). Although Architecture Related Posts (ARPs) communicate architectural information that has a system-wide impact on development, little is known about how SO users revise information shared in ARPs. To fill this gap, we conducted an empirical study to understand how users revise Architecture Related Questions (ARQs) on SO. We manually checked 13,205 ARPs and finally identified 4,114 ARQs that contain revision information. Our main findings are that: (1) The revision of ARQs is not prevalent in SO, and an ARQ revision starts soon after this question is posted (i.e., from 1 minute onward). Moreover, the revision of an ARQ occurs before and after this question receives its first answer/architecture solution, with most revisions beginning before the first architecture solution is posted. Both Question Creators (QCs) and non-QCs actively participate in ARQ revisions, with most revisions being made by QCs. (2) A variety of information (14 categories) is missing and further provided in ARQs after being posted, among which design context, component dependency, and architecture concern are dominant information. (3) Clarify the understanding of architecture under design and improve the readability of architecture problem are the two major purposes of the further provided information in ARQs. (4) The further provided information in ARQs has several impacts on the quality of answers/architecture solutions, including making architecture solution useful, making architecture solution informative, making architecture solution relevant, among others.

Problem

• How users revise architectural questions on Stack Overflow
• Identify missing information in architecture-related questions
• Impact of revisions on answer quality and usefulness

Innovation

• Empirical study on ARQ revisions
• Manual analysis of 4114 ARQs
• Identified 14 missing information categories