Title: Barry and the BAO Model Comparison

Abstract: We compare the performance of four state-of-the-art models for extracting isotropic measurements of the Baryon Acoustic Oscillation (BAO) scale. To do this, we created a new, public, modular code Barry, which contains datasets, model fitting tools, and model implementations incorporating different descriptions of non-linear physics and algorithms for isolating the BAO feature. These are then evaluated for bias, correlation, and fitting strength using mock power spectra and correlation functions developed for the Sloan Digital Sky Survey Data Release 12. Our main findings are as follows: 1) All of the models can recover unbiased constraints when fit to the pre- and post-reconstruction simulations. 2) Models that provide physical descriptions of the damping of the BAO feature (using e.g., standard perturbation or effective-field theory arguments) report smaller errors on average, although the distribution of mock $\chi^{2}$ values indicates these are underestimated. 3) Allowing the BAO damping scale to vary can provide tighter constraints for some mocks, but is an artificial improvement that only arises when noise randomly sharpens the BAO peak. 4) Unlike recent claims in the literature when utilising a BAO Extractor technique, we find no improvement in the accuracy of the recovered BAO scale. 5) We implement a procedure for combining all models into a single consensus result that improves over the standard method without obviously underestimating the uncertainties. Overall, Barry provides an excellent framework for performing the cosmological analyses for upcoming surveys, and for rapidly testing and validating new models.