Understanding How Paper Writers Use AI-Generated Captions in Figure Caption Writing

2025-01-10

AI Summary

This study investigates how AI can effectively assist scholarly authors in generating high-quality figure captions. Method: Through a user study with 18 authors who rewrote captions for their own published figures, we collected screen recordings, coded interaction behaviors, and conducted qualitative analysis—providing the first author-centered empirical account of adoption, editing, and rejection patterns toward AI-generated captions. Initial captions were produced using state-of-the-art multimodal models. Results: Authors predominantly employed a “copy-and-refine” strategy; caption length, information density, and image-text alignment emerged as key determinants of adoption. We also identified persistent limitations of current AI systems in captioning complex scientific figures. Crucially, our findings empirically confirm author preference for longer, integrative captions that explicitly bridge visual and textual content. Based on these insights, we propose design principles for AI tools supporting closed-loop academic writing—offering actionable, evidence-based guidance for developing scholarly AI writing assistants.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Figures and their captions play a key role in scientific publications. However, despite their importance, many captions in published papers are poorly crafted, largely due to a lack of attention by paper authors. While prior AI research has explored caption generation, it has mainly focused on reader-centered use cases, where users evaluate generated captions rather than actively integrating them into their writing. This paper addresses this gap by investigating how paper authors incorporate AI-generated captions into their writing process through a user study involving 18 participants. Each participant rewrote captions for two figures from their own recently published work, using captions generated by state-of-the-art AI models as a resource. By analyzing video recordings of the writing process through interaction analysis, we observed that participants often began by copying and refining AI-generated captions. Paper writers favored longer, detail-rich captions that integrated textual and visual elements but found current AI models less effective for complex figures. These findings highlight the nuanced and diverse nature of figure caption composition, revealing design opportunities for AI systems to better support the challenges of academic writing.

Problem

• Artificial Intelligence
• Scientific Writing Assistance
• Figure Caption Optimization

Innovation

• AI-generated captions
• enhanced figure captions
• text-image integration