How Well Can AI Generate Backlogs from App Mockups?

2026-07-24

AI Summary

This study addresses the inefficiency and incompleteness of traditional sprint backlog creation, particularly during early project stages when structured input is scarce. The authors propose a multimodal approach that automatically generates backlogs—including epics, user stories, and tasks—from application interface mockups by jointly reasoning over visual and textual information. Their method innovatively integrates Compositional Chain-of-Thought (CCoT) reasoning with role-driven prompting, further enhanced by architectural context to improve generation quality. They also introduce a “corrected recall” metric incorporating developer judgments to better reflect practical utility. Experimental results show that CCoT achieves F1 scores of 52–66% on epics and user stories; integrating architectural context boosts backend task precision by up to 35%; notably, 26% of false positives were still deemed useful by developers.

Abstract

Creating sprint backlogs requires considerable effort, as items such as epics, user stories, and tasks can be missed or inconsistently specified. We propose a multimodal approach to support backlog generation from visual app mockups, an artifact available at early project stages. We evaluate three prompting strategies on GPT-4o: a zero-shot baseline, Compositional Chain-of-Thought (CCoT) for vision-language reasoning, and a persona-driven prompt. We study seven app development projects across two countries and interview developers about the results. Overall, we observed that the baseline prompt favours recall over precision, whereas CCoT is more balanced, achieving average F1 scores of 52-66% for epics and user stories. Tasks were more challenging to generate accurately. Precision gains were most consistent when adding architectural context, particularly for backend tasks (precision gains up to 35%). Interviews with developers revealed that up to 26% of false positives were still considered useful, reflecting the creative and open-ended nature of backlog creation. To capture this, we propose a new measure called Revised Recall, which complements ground-truth evaluation with developer assessments. Our findings suggest that hybrid prompting with architectural context can assist backlog generation from early mockups, though results vary by item type and developer oversight remains necessary.

Problem

• backlog generation
• app mockups
• user stories
• agile development
• AI-assisted software engineering

Innovation

• multimodal backlog generation
• Compositional Chain-of-Thought
• architectural context
• Revised Recall
• GPT-4o prompting