Title: Squeezing of nonlinear spin observables by one axis twisting in the presence of decoherence: An analytical study

Abstract: In an ensemble of two-level atoms that can be described in terms of a collective spin, entangled states can be used to enhance the sensitivity of interferometric precision measurements. While non-Gaussian spin states can produce larger quantum enhancements than spin-squeezed Gaussian states, their use requires the measurement of observables that are nonlinear functions of the three components of the collective spin. In this paper we develop strategies that achieve the optimal quantum enhancements using non-Gaussian states produced by a nonlinear one-axis-twisting Hamiltonian, and show that measurement-after-interaction techniques, known to amplify the output signals in quantum parameter estimation protocols, are effective in measuring nonlinear spin observables. Including the presence of the relevant decoherence processes from atomic experiments, we determine analytically the quantum enhancement of non-Gaussian over-squeezed states as a function of the noise parameters for arbitrary atom numbers.