Title: Analytical Modeling of Nonlinear Fiber Propagation for Four Dimensional Symmetric Constellations

Abstract: Coherent optical transmission systems naturally lead to a four dimensional (4D) signal space, i.e., two polarizations each with two quadratures. In this paper we derive an analytical model to quantify the impact of Kerr nonlinearity on such 4Dspaces, taking the interpolarization dependency into account. This is in contrast to previous models such as the GN and EGN models, which are valid for polarization multiplexed (PM)formats, where the two polarizations are seen as independent channels on which data is multiplexed. The proposed model agrees with the EGN model in the special case of independent two-dimensional modulation in each polarization. The model accounts for the predominant nonlinear terms in a WDM system, namely self-phase modulation and and cross-phase modulation. Numerical results show that the EGN model may inaccurately estimate the nonlinear interference of 4D formats. This nonlinear interference discrepancy between the results of the proposed model and the EGN model could be up to 2.8 dB for a system with 80 WDM channels. The derived model is validated by split-step Fourier simulations, and it is shown to follow simulations very closely.