Title: Gaussian process hyper-parameter estimation using parallel asymptotically independent Markov sampling

Abstract: Gaussian process surrogates of computationally expensive computer codes provide fast statistical approximations to model physical processes. The training of these surrogates depends on the set of design points chosen to run the expensive simulator. However, such training set is bound to be limited and quantifying the resulting uncertainty in the hyper-parameters of the emulator by uni-modal distributions is likely to induce bias. This paper proposes a computationally efficient sampler based on an extension of the Asymptotically Independent Markov Sampling, a recently developed algorithm for Bayesian inference and extended to optimisation problems. Structural uncertainty of the emulator is obtained as a by-product of the Bayesian treatment of the hyper-parameters. Model uncertainty is also acknowledged through numerical stabilisation measures by including a nugget term in the formulation of the probability model. The efficiency of the proposed sampler is illustrated in examples where multi-modal distributions are encountered. For the purposes of reproducibility, further development, and use in other applications, we made the code used to generate the examples in this paper freely available for download at this https URL