Title: Non-tensorial Gravitational Wave Background in NANOGrav 12.5-Year Data Set

Abstract: We perform the first search for an isotropic non-tensorial gravitational-wave background (GWB) allowed in general metric theories of gravity in the North American Nanohertz Observatory for Gravitational Waves (NANOGrav) 12.5-year data set. By modeling the GWB as a power-law spectrum, we find strong Bayesian evidence for a spatially correlated process with scalar transverse (ST) correlations whose Bayes factor versus the spatially uncorrelated common-spectrum process is $99\pm 7$, but no statistically significant evidence for the tensor transverse, vector longitudinal and scalar longitudinal polarization modes. The median and the $90\%$ equal-tail amplitudes of ST mode are $\mathcal{A}_{\mathrm{ST}}= 1.06^{+0.35}_{-0.28} \times 10^{-15}$, or equivalently the energy density parameter per logarithm frequency is $\Omega_{\mathrm{GW}}^{\mathrm{ST}} = 1.54^{+1.20}_{-0.71} \times 10^{-9}$, at frequency of 1/year.