The ICSE 2026 Shadow PC: Training the Next Generation of Reviewers Through Deliberate Practice

2026-07-22

AI Summary

Peer review is critical in software engineering research, yet systematic reviewer training has long been lacking. This work proposes and implements a large-scale Shadow Program Committee (Shadow PC) training initiative for ICSE 2026, featuring multi-stage deliberate practice, structured calibration exercises, and peer feedback mechanisms, all while maintaining strict separation from the main Program Committee to ensure review independence. Innovatively, the initiative introduces Shadow PC Area Chairs to establish a sustainable leadership development pipeline. With 102 participants completing reviews for 117 papers, 97% of participants recommended the experience, and 67% of authors found the reviews helpful, demonstrating the feasibility of scaling high-quality reviewer training effectively.

Abstract

Peer review is essential to software engineering research, yet reviewer training remains largely implicit. We describe the ICSE 2026 Shadow PC, a redesigned program emphasizing deliberate practice at scale. Key innovations include multi-phase structure with calibration and peer feedback, strict separation from the main PC, and a pathway toward leadership development. With 102 participants completing the program and reviewing 117 papers, 97% recommending the experience, and positive reception from authors (67% finding reviews helpful), the program demonstrates that rigorous reviewer training is achievable at scale. We share lessons learned and propose shadow PC area chairs as a mechanism for sustainable scaling and leadership development.

Problem

• peer review
• reviewer training
• software engineering research
• deliberate practice
• shadow program

Innovation

• deliberate practice
• shadow program committee
• reviewer training
• peer feedback
• leadership development