🤖 AI Summary
This study challenges the “intellectual fairness” of Italy’s national research evaluation panels in economics, statistics, and business—questioning whether surface-level demographic balance (e.g., gender, institutional affiliation) conceals deeper intellectual homogeneity. Method: We propose a multidimensional academic network framework—integrating co-authorship, journal publication, and institutional affiliation—to quantify panel members’ epistemic connectivity and cognitive diversity via social network analysis. Using graph-theoretic metrics (network density, clustering coefficient) and a randomized controlled design, we compare official panels against randomly sampled counterparts across disciplinary domains. Contribution/Results: Both officially appointed panels exhibit significantly higher network connectivity than their random counterparts, revealing structural intellectual homogeneity that undermines substantive fairness in peer review. The study shifts beyond demographic proxies for fairness, introducing a rigorous, operationalizable metric for assessing intellectual diversity in research governance.
📝 Abstract
Research evaluation is usually governed by panels of peers. Procedural fairness refers to the principles that ensures decisions are made through a fair and transparent process. It requires that the composition of panels is fair. A fair panel is usually defined in terms of observable characteristics of scholars such as gender or affiliations. The formal adherence to these criteria is not sufficient to guarantee a fair composition in terms of scholarly thinking, background, or policy orientation. An empirical strategy for exploring the fairness in the intellectual composition of panels is proposed, based on the observation of links between panellists. The case study regards the three panels selected to evaluate research in economics, statistics and business during the Italian research assessment exercises. The first two panels were appointed directly by the governmental agency responsible for the evaluation, while the third was randomly selected. Hence the third panel can be considered as a control for evaluating about the fairness of the others. The fair representation is explored by comparing the networks of panellists based on their co-authorship relations, the networks based on journals in which they published and the networks based on their affiliated institutions (universities, research centres and newspapers). The results show that the members of the first two panels had connections much higher than the members of the control group. Hence the composition of the first two panels should be considered as unfair, as the results of the research assessments.