I am happy to announce that our paper “Statistical Models for the Analysis of Optimization Algorithms with Benchmark Functions” by David Issa Mattos, Jan Bosch and Helena Holmström Olsson has been accepted for publication at the IEEE Transactions on Evolutionary Computation (DOI:10.1109/TEVC.2021.3081167)!

The process took roughly 10 months with 3 revisions and has been accepted this week.

The abstract follows below and the pre-print can be found at https://arxiv.org/pdf/2010.03783.pdf

Frequentist statistical methods, such as hypothesis testing, are standard practices in studies that provide benchmark comparisons. Unfortunately, these methods have often been misused, e.g., without testing for their statistical test assumptions or without controlling for family-wise errors in multiple group comparisons, among several other problems. Bayesian Data Analysis (BDA) addresses many of the previously mentioned shortcomings but its use is not widely spread in the analysis of empirical data in the evolutionary computing community. This paper provides three main contributions. First, we motivate the need for utilizing Bayesian data analysis and provide an overview of this topic. Second, we discuss the practical aspects of BDA to ensure that our models are valid and the results are transparent. Finally, we provide five statistical models that can be used to answer multiple research questions. The online appendix provides a step-by-step guide on how to perform the analysis of the models discussed in this paper, including the code for the statistical models, the data transformations, and the discussed tables and figures.