mixed-methods analysis

Designs and conducts studies and evaluations that combine quantitative and qualitative data, building mixed-methods study designs, integrated datasets and analysis pipelines, joint displays and triangulation matrices to synthesize findings. Develops and applies integration and interpretation strategies (e.g., qualitative–quantitative synthesis, phenomenon‑based mixed methods) to reconcile convergent and divergent results and produce coherent inferences and recommendations.

mixed-methodsanalysis

Recent Skill Trend

Momentum and market value over time
Trending
Score
No comparison yet
-0.09
Oct 01, 2026Oct 01, 2026
Career
Value
No comparison yet
$184K/year
Oct 01, 2026Oct 01, 2026

Must-Read Papers

Most classic and influential ideas
View more

Combining an experimental study with external data: study designs and identification strategies

Jun 05, 2024
LU
Lawson Ung
🏛️ Harvard T.H. Chan School of Public Health | Dartmouth Geisel School of Medicine | Beth Israel Deaconness Medical Center

This study addresses the challenge of integrating randomized or single-arm clinical trials with external experimental or observational data to enable cross-study treatment comparisons and improve estimation precision of treatment effects. Methodologically, building upon the potential outcomes framework, we first develop a unified identification strategy for hybrid-data designs, systematically characterizing identifiability conditions across diverse designs—including historical controls, synthetic controls, and anchoring estimators—and propose a generalizable taxonomy of such designs along with corresponding causal inference principles. Our contribution lies in filling a critical theoretical gap in regulatory science regarding the rigorous integration of external controls, thereby establishing a methodological foundation for leveraging real-world evidence to complement trial-based evidence in pharmaceutical and medical device evaluation. This advancement significantly enhances the transportability of evidence and its applicability to regulatory decision-making.

Combining experimental studies with external data sourcesDeveloping identification strategies for treatment effectsFormalizing study designs to support systematic evaluation

Existing computational tools for qualitative data analysis often fall short in effectively supporting causal exploration due to insufficient contextual awareness, limited trustworthiness, or overly complex outputs. To address these limitations, this work proposes QualCausal, the first interactive causal analysis system grounded in user research–driven design principles. Developed through formative user studies, QualCausal integrates context-aware processing, cognitive scaffolding, and explainability mechanisms to facilitate efficient exploration and validation of causal hypotheses within qualitative datasets. The system enables researchers to extract causal relationships, construct interactive causal networks, and examine findings through coordinated multi-view visualizations. User evaluations demonstrate that QualCausal significantly reduces analytical burden, provides robust cognitive support, and prompts critical reflection on how computational tools can be meaningfully integrated into social science research practices, thereby bridging the gap between computational assistance and qualitative inquiry paradigms.

causal relationshipscomputational toolscontext

Qualitative Study for LLM-assisted Design Study Process: Strategies, Challenges, and Roles

Jul 14, 2025
SR
Shaolun Ruan
🏛️ Singapore Management University | Monash University | Hong Kong University of Science and Technology | Nanyang Technological University | Zhejiang University

Despite growing adoption of large language models (LLMs) in visualization design research, there remains a lack of systematic empirical understanding of their practical roles and limitations. Method: We conducted a multi-stage qualitative study—including in-depth interviews and structured surveys—with 30 interdisciplinary researchers actively using LLMs in real-world visualization projects. Contribution/Results: We identify LLMs’ concrete functions, recurrent strategies, and shared challenges across key design phases—problem framing, data comprehension, and solution generation. Building on these insights, we propose “VizLLM,” the first comprehensive application framework that systematically integrates LLM-assisted mechanisms and evidence-informed practice principles throughout the end-to-end visualization design process. This work bridges a critical theoretical gap in LLM-augmented design research and delivers a reusable, empirically grounded methodology with actionable implementation pathways for researchers and practitioners.

How LLMs enhance visualization design studiesRoles of LLMs in design study stagesStrategies and challenges in LLM-assisted design

This study examines the tension between efficiency gains and researcher autonomy arising from AI-assisted analysis in qualitative research. Through in-depth interviews with 16 qualitative researchers, it comparatively analyzes acceptance and underlying mechanisms across three coding paradigms: fully manual, human-initiated AI-assisted, and AI-initiated. The study innovatively conceptualizes AI explicitly as a “supporter”—neither collaborator nor supervisor—and identifies three core determinants of adoption: efficiency enhancement, attribution of interpretive ownership, and algorithmic trust. Findings indicate broad acceptance of AI for accelerating coding and thematic analysis, yet strong consensus on human primacy in meaning-making and interpretive authority. Enhancing procedural transparency, researcher control, and structured human–AI collaboration significantly strengthens trust and mitigates bias risks. The work provides theoretical grounding and actionable guidelines for developing human-centered, accountable AI-augmented qualitative research workflows.

Addressing concerns about AI's contextual understanding in interpretive researchBalancing AI efficiency with researcher ownership in qualitative data analysisInvestigating researcher preferences for AI delegation levels in coding processes

Handling Missingness, Failures, and Non-Convergence in Simulation Studies: A Review of Current Practices and Recommendations

Sep 27, 2024
SP
Samuel Pawel
🏛️ University of Zurich | University of Amsterdam | University of Marburg | Leiden University Medical Center | Ernst-Abbe University of Applied Sciences Jena

Accurately evaluating the performance of analytical methods in simulation studies is hindered by underreporting and inconsistent handling of “missingness” issues—such as algorithm failure or non-convergence—that compromise validity and reproducibility. Method: We conducted a large-scale empirical analysis of 482 methodological simulation studies, systematically extracting metadata, applying qualitative coding, and performing case studies—including publication bias correction—to quantify the prevalence and reporting practices of missingness. Contribution/Results: We found that only 23% of studies mentioned missingness and merely 14% described mitigation strategies. Based on these findings, we developed a novel missingness taxonomy tailored to simulation research and proposed actionable principles—including mandatory missingness reporting—alongside a comprehensive, end-to-end practice guideline covering reporting, handling, and replication. Validation confirmed substantial improvements in transparency, comparability, and reproducibility of simulation studies.

Addressing missing data in simulation studies due to failures.Providing solutions for non-convergence in methodological research.Recommending practices to report and handle simulation missingness.

Latest Papers

What's happening recently
View more

Traditional questionnaires struggle to simultaneously capture qualitative depth and quantitative structure, limiting comprehensive understanding of complex social phenomena. This study proposes a dynamic survey platform powered by large language models (LLMs) that, for the first time, enables real-time semantic clustering of open-ended responses. Through an interactive feedback mechanism, users can rate, rank, and reflect on these clusters, generating visual reports that integrate qualitative insights with quantitative analysis. Innovatively embedding LLMs within a closed-loop data collection framework, the approach facilitates dynamic comparisons between individual perspectives and group-level trends. Empirical validation across two field studies involving 93 participants demonstrates that the platform significantly enhances data richness and user engagement compared to conventional survey tools, while effectively fostering collaborative sensemaking.

collaborative interactionLLMsqualitative depth

This study addresses the lack of systematic understanding regarding the types and motivations of visual representations in qualitative research. Building upon and extending Verdinelli & Scagnoli’s (2013) work through a data-driven literature review, it conducts a content analysis of articles and their visualizations published between 2020 and 2022 in three leading qualitative methods journals. Integrating epistemological stance classification with visualization-type coding, the study innovatively combines correspondence analysis and cognitive network analysis for the first time. Findings indicate that while visualizations remain underutilized in qualitative research, their typological diversity is increasing, and the choice of graphical representation appears largely independent of the authors’ epistemological positions. These results offer both empirical grounding and methodological innovation for integrating interdisciplinary visualization tools into qualitative inquiry.

epistemologymedia representationqualitative research

This study addresses the challenge of ensuring rigor in causal inference under multi-source heterogeneous data fusion by proposing a structured design paradigm grounded in the target trial framework. The approach explicitly incorporates the target population and its sampling model into the causal analysis, systematically integrating external controls, generalizability, and transportability assessments through data element alignment, transparent assumption articulation, and emulation of the target trial. Its key innovation lies in anchoring the entire framework to a precise definition of the target population, thereby identifying and mitigating irreconcilable conflicts across data sources. This strategy enhances both the reliability and interpretability of causal conclusions derived from complex, real-world data ecosystems.

causal inferencedata integrationexternal comparator analyses

This study explores the effective integration of large language models (LLMs) into qualitative and mixed-methods social network analysis to augment—rather than replace—the deep analytical capacities of human researchers. Focusing on core issues such as relational meaning, narrative interpretation, and identity construction, the work proposes an LLM application paradigm oriented toward enhancing methodological rigor. This paradigm emphasizes human–AI collaboration, reflexive practice, and ethical accountability. By incorporating LLM-assisted data coding, theory generation, and abductive reasoning, the project develops a methodologically innovative yet practically feasible framework for qualitative social network analysis. The approach significantly improves analytical efficiency while upholding scholarly standards and ethical compliance.

Ethical ChallengesLarge Language ModelsMixed-Methods Research

A Design Study Process Model for Medical Visualization

Dec 24, 2025
MF
Mengjie Fan
🏛️ Peking University

Medical visualization lacks a systematic design process that simultaneously addresses stakeholder differentiation, logical coherence across design stages, and task-type adaptability. Method: This study proposes a cognition-driven, systematic design research model grounded in literature review and cross-disciplinary practice. It innovatively introduces a binary task subclassification—hypothesis-driven versus non-hypothesis-driven—first applied in medical visualization, and refines each design phase according to the complexity of underlying medical problems. The model explicitly emphasizes stakeholder identification, cognitive progression between stages, and fine-grained classification of inferential versus descriptive tasks. Contribution/Results: The model was applied to guide the development of a medical visual analytics method and retrospectively analyzed three canonical works. Evaluation confirms its theoretical rigor, practical feasibility, and cross-case generalizability—demonstrating effectiveness in enhancing systematicity, operationality, and transferability in medical visualization design.

Develops a design study process model for medical visualization research.Emphasizes stakeholder distinction and task classification in design stages.Provides systematic guidance for interdisciplinary medical visualization applications.

Hot Scholars

AS

Anita Sarma

Oregon State University
Software EngineeringHuman Computer InteractionDistributed Software Development
SM

Stephen MacNeil

Assistant Professor, Temple University
Human-computer InteractionGenerative AIDesignComputing Education
PD

Paul Denny

Professor, University of Auckland
Educational technologyComputer Science Education
MK

Marcos Kalinowski

Professor, Pontifical Catholic University of Rio de Janeiro (PUC-Rio)
Empirical Software EngineeringAI EngineeringAI4SEHuman Aspects in Software Engineering
SG

Shion Guha

University of Toronto
Human-Centered Data SciencePublic Interest TechnologyResponsible AIAI Policy