Title: Towards a self-consistent analysis of the anisotropic galaxy two- and three-point correlation functions on large scales: application to mock galaxy catalogues

Abstract: We establish a practical method for the joint analysis of anisotropic galaxy two- and three-point correlation functions (2PCF and 3PCF) on the basis of the decomposition formalism of the 3PCF using tri-polar spherical harmonics. We perform such an analysis with MultiDark Patchy mock catalogues to demonstrate and understand the benefit of the anisotropic 3PCF. We focus on scales above $80 h^{-1}\,{\rm Mpc}$, and use information from the shape and the baryon acoustic oscillation (BAO) signals of the 2PCF and 3PCF. We also apply density field reconstruction to increase the signal-noise ratio of BAO in the 2PCF measurement, but not in the 3PCF measurement. In particular, we study in detail the constraints on the angular diameter distance and the Hubble parameter. We build a model of the bispectrum or 3PCF that includes the nonlinear damping of the BAO signal in redshift space. We carefully account for various uncertainties in our analysis including theoretical models of the 3PCF, window function corrections, biases in estimated parameters from the fiducial values, the number of mock realizations to estimate the covariance matrix, and bin size. The joint analysis of the 2PCF and 3PCF monopole and quadrupole components shows a $30\%$ a nd $20\%$ improvement in Hubble parameter constraints before and after reconstruction of the 2PCF measurements, respectively, compared to the 2PCF analysis alone. This study clearly shows that the anisotropic 3PCF increases cosmological information from galaxy surveys and encourages further development of the modeling of the 3PCF on smaller scales than we consider.