🤖 AI Summary
This study addresses the longstanding limitation in networking research caused by the scarcity of reliable and reproducible experimental data. To overcome this challenge, the authors propose employing high-fidelity software models as substitutes for physical devices, enabling the construction of reproducible validation environments through “natural experiments” and establishing a reproducibility-based criterion for experimental reliability. A systematic evaluation framework is developed to conduct full-scale assessments of mainstream network simulation tools. The findings reveal that while most tools technically satisfy reproducibility requirements, their adoption in practice is predominantly influenced by non-technical factors such as popularity and user familiarity. This work introduces a new paradigm and benchmark for reproducible experimentation in networking research.
📝 Abstract
Background: When we model networks, there is a problem of obtaining experimental data to verify other model approaches. And even if there are some experimental data, it is necessary to be sure of their reliability. Purpose: It is necessary to propose methods for obtaining reliable experimental data. Method: By its nature, network equipment is a software and hardware complex. Therefore, a full-scale software model can be considered completely equivalent to real equipment. And a real experiment can be replaced by a nature experiment. The reliability of a nature experiment will be based on its reproducibility. Results A comparison of popular nature network modeling packages was carried out. These packages were divided by functionality and feasibility of reproducible studies. Conclusions: Most software packages meet the reproducibility criteria. The choice of a specific solution depends on non-technical factors: popularity and knowledge of the package.