Guiding Principles for Using Mixed Methods Research in Software Engineering

2024-04-09

AI Summary

Software engineering researchers often lack formal training in social science methodologies, leading to suboptimal adoption and implementation of mixed-methods designs. Method: Drawing on social science research design theory, empirical software engineering paradigms, and multi-case contextual analysis, this paper systematically develops a practical guide for designing and conducting mixed-methods studies in software engineering. Contribution/Results: It introduces— for the first time in software engineering—the three core mixed-methods design paradigms, identifies and warns against prevalent methodological anti-patterns, and clarifies the boundary conditions and epistemic prerequisites under which mixed methods offer added value over single-method approaches. The paper proposes an actionable decision framework supporting design selection, trade-off analysis, and context-sensitive methodological judgment. This framework enables rigorous, intentional, and paradigm-crossing empirical research, substantially lowering methodological barriers while enhancing the scientific rigor and innovative potential of mixed-methods studies in software engineering.

Technology Category

• Game Theory and Economic Paradigms: Mechanism Design
• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Social Networks and Social Media: Computational social science

Abstract

Mixed methods research is often used in software engineering, but researchers outside of the social or human sciences often lack experience when using these designs. This paper provides guiding principles and advice on how to design mixed method research, and to encourage the intentional, rigorous, and innovative application of mixed methods in software engineering. It also presents key properties of core mixed method research designs. Through a number of fictitious but recognizable software engineering research scenarios, we showcase how to choose suitable designs and consider the inevitable trade-offs any design choice leads to. We describe several antipatterns that illustrate what to avoid in mixed method research, and when mixed method research should be considered over other approaches.

Problem

• Guiding principles for mixed methods research
• Designing rigorous software engineering studies
• Avoiding antipatterns in research methodologies

Innovation

• Guiding mixed methods design
• Showcasing research scenarios
• Describing antipatterns avoidance