Curating Model Problems for Software Designing

2025-03-09

AI Summary

The software design community lacks a consensus-based, multi-perspective analyzable benchmark problem repository, hindering comparability and reproducibility in pedagogy and empirical research. To address this, we propose the first collaborative model-problem co-construction framework for software design education and research. Our method integrates participatory workshops, a structured problem template, explicit mapping to educational scenarios, and iterative consensus refinement to systematically build an open, reusable, and extensible benchmark problem set. Key contributions include: (1) a community-driven, sustainably evolving framework; (2) standardized problem representation dimensions enabling cross-study comparison; and (3) an initial release covering canonical design tasks—already deployed in university courses and tool evaluations. This work establishes the first open, pedagogically grounded, and empirically usable benchmark suite for software design, directly addressing a critical gap in both teaching practice and research infrastructure.

Technology Category

• Constraint Satisfaction and Optimization: Solvers and Tools
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Application Domains: Software Engineering

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

Many disciplines use standard examples for education and to share and compare research results. The examples are rich enough to study from multiple points of view; they are often called model problems. Software design lacks such a community resource. We propose an activity for Designing 2025 in which participants improve some existing model problem descriptions and initiate new ones -- with a focus on use in software design education, plus potential utility in research.

Problem

• Lack of standard examples in software design education
• Need for model problems to compare research results
• Proposal to create and improve model problems for 2025

Innovation

• curating model problems
• improving existing descriptions
• focusing on software education