Statistics for Ranking Program Committees and Editorial Boards

πŸ“… 2010-02-04
πŸ›οΈ arXiv.org
πŸ“ˆ Citations: 6
✨ Influential: 0
πŸ“„ PDF

career value

217K/year
πŸ€– AI Summary
Existing academic impact metrics lack organizational-level adaptability, particularly for research teams such as conference program committees (PCs) or journal editorial boards. Method: This paper proposes the alpha-indexβ€”a novel composite metric that jointly models statistical homogeneity (extending group h-index consistency) and collective h-group influence, moving beyond simple aggregation of individual h-indices. It is the first to unify homogeneity measurement with an extended group h-index framework. Results: Empirical evaluation in computer science demonstrates that alpha-index rankings of conference PCs align strongly with authoritative manual classifications (Pearson *r* > 0.85), significantly outperforming baselines including h-median and h-sum. The metric offers an interpretable, reproducible, and organization-aware assessment paradigm, directly supporting high-stakes decision-making in research funding allocation, project review, and editorial board selection.
πŸ“ Abstract
Ranking groups of researchers is important in several contexts and can serve many purposes such as the fair distribution of grants based on the scientist's publication output, concession of research projects, classification of journal editorial boards and many other applications in a social context. In this paper, we propose a method for measuring the performance of groups of researchers. The proposed method is called alpha-index and it is based on two parameters: (i) the homogeneity of the h-indexes of the researchers in the group; and (ii) the h-group, which is an extension of the h-index for groups. Our method integrates the concepts of homogeneity and absolute value of the h-index into a single measure which is appropriate for the evaluation of groups. We report on experiments that assess computer science conferences based on the h-indexes of their program committee members. Our results are similar to a manual classification scheme adopted by a research agency.
Problem

Research questions and friction points this paper is trying to address.

Measuring group performance using h-index and homogeneity
Integrating h-group concept with researcher homogeneity metrics
Evaluating scientific groups for fair resource allocation
Innovation

Methods, ideas, or system contributions that make the work stand out.

Combines h-index homogeneity with group evaluation
Extends h-index concept to research groups
Integrates absolute h-value with group homogeneity
R
Roberto da Silva
Instituto de FΓ­sica, Universidade Federal do Rio Grande do Sul
J
J. Oliveira
Instituto de Informatica, Universidade Federal do Rio Grande do Sul
J
J. V. D. Lima
Instituto de Informatica, Universidade Federal do Rio Grande do Sul
V
V. Moreira
Instituto de Informatica, Universidade Federal do Rio Grande do Sul