Title: Bayesian Fixed-domain Asymptotics: Bernstein-von Mises Theorem for Covariance Parameters in a Gaussian Process Model

Abstract: Gaussian process models typically contain finite dimensional parameters in the covariance functions that need to be estimated from the data. We study the Bayesian fixed-domain asymptotic properties of the covariance parameters in a Gaussian process with an isotropic Matern covariance function. Under fixed-domain asymptotics, it is well known that when the dimension of data is less than or equal to three, the microergodic parameter can be consistently estimated with asymptotic normality while the variance parameter and the range (or length-scale) parameter cannot. Motivated by the frequentist theory, we prove a Bernstein-von Mises theorem for the covariance parameters in isotropic Matern covariance functions. We show that under fixed-domain asymptotics, the joint posterior distribution of the microergodic parameter and the range parameter can be factored independently into the product of their marginal posteriors as the sample size goes to infinity. The posterior of the microergodic parameter converges in total variation norm to a normal distribution with shrinking variance, while the posterior of the range parameter does not necessarily converge to any degenerate distribution in general. Our theory allows unbounded prior support for the range parameter. Furthermore, we propose a new property called the posterior asymptotic efficiency in linear prediction, and show that the Bayesian kriging predictor at a new spatial location with covariance parameters randomly drawn from their posterior has the same prediction mean squared error as if the true parameters were known. In the special case of one-dimensional Ornstein-Uhlenbeck process, we derive an explicit form for the limiting posterior distribution of the range parameter and an explicit posterior convergence rate for the posterior asymptotic efficiency. We verify these asymptotic results in numerical examples.