Title: Pure polarization test of GW170814 and GW170817 using waveforms consistent with modified theories of gravity

Abstract: The physical degrees of freedom of a gravitational wave (GW) are imprints of the nature of gravity. We can test a gravity theory by searching for polarization modes beyond general relativity. The LIGO-Virgo collaboration analyzed several GW events in the O1 and O2 observing runs in the pure polarization framework, where they perform the Bayesian model selection between general relativity and the theory allowing only scalar or vector polarization modes. In this paper, we reanalyze the polarizations of GW170814 (binary black hole merger) and GW170817 (binary neutron star merger) in the improved framework of pure polarizations including the angular patterns of nontensorial radiation. We find logarithms of the Bayes factors of 2.775 and 3.636 for GW170814 in favor of the pure tensor polarization against pure vector and scalar polarizations, respectively. These Bayes factors are consistent with the previous results by the LIGO-Virgo collaboration, though the estimated parameters of the binaries are significantly biased. For GW170817 with the priors on the location of the binary from NGC4993, we find logarithms of the Bayes factors of 21.078 and 44.544 in favor of the pure tensor polarization against pure vector and scalar polarizations, respectively. These more strongly support GR, epecially compared to the scalar polarization, than the previous results by the LIGO-Virgo collaboration due to the location prior. In addition, by utilizing the orientation information of the binary from a gamma-ray burst jet, we find logarithms of the Bayes factor of 51.043 and 60.271 in favor of the pure tensor polarization against pure vector and pure scalar polarization, much improved from those without the jet prior.