Introducing precision-weighted bias as a performance measure to inform the inclusion of adaptive designs in meta-analysis

2026-06-10

AI Summary

Current systematic review guidelines restrict the inclusion of adaptive designs due to concerns about bias, yet they overlook the joint influence of bias and information content. This work proposes “precision-weighted bias” as a novel metric, which accounts for each study’s unconditional bias scaled by its precision, thereby more accurately reflecting its contribution to overall meta-analytic bias. Theoretical derivation demonstrates that the total bias in a meta-analysis is in fact a precision-weighted average of individual study biases, rather than a simple arithmetic mean. Simulation studies further reveal that although adaptive designs may exhibit unweighted bias, their precision-weighted bias is often negligible—resulting in minimal impact on pooled estimates when included. These findings provide both theoretical justification and methodological support for the appropriate inclusion of adaptive designs in systematic reviews.

Abstract

We propose a novel, intuitive measure of statistical performance: precision-weighted bias. Precision-weighted bias is defined as the unconditional bias of an estimator weighted by the degree of information (precision) it contains. Current guidelines, such as GRADE and CONSORT, often view the potential for increased bias in adaptive designs as a deterrent for the inclusion of such designs in systematic reviews. However, we demonstrate that the bias in a common-effect meta-analysis is approximately equal to the precision-weighted average of the precision-weighted biases of its constituent studies, rather than of their unweighted unconditional biases. Through simulation studies, we show that while adaptive designs may exhibit unweighted bias, they frequently have zero precision-weighted bias. Consequently, including these designs often results in a negligible change to the overall meta-analysis bias. These results suggest that precision-weighted bias is a superior indicator for determining whether to include an adaptive design in a meta-analysis. We recommend that precision-weighted bias be used as a standard complement to unweighted unconditional and conditional bias in simulation studies to support more inclusive and accurate evidence synthesis.

Problem

• adaptive designs
• meta-analysis
• bias
• precision-weighted bias
• evidence synthesis

Innovation

• precision-weighted bias
• adaptive designs
• meta-analysis
• statistical performance
• evidence synthesis