Title: Analysis and Experimental Demonstration of Orthant-Symmetric Four-dimensional 7 bit/4D-sym Modulation for Optical Fiber Communication

Abstract: We propose a new four-dimensional orthant-symmetric 128-ary modulation format (4D-OS128) with a spectral efficiency of 7bit/4D-sym. The proposed format fills the gap between polarization-multiplexed 8- and 16-ary quadrature-amplitude modulation (PM-8QAM and PM-16QAM).Numerical simulations show that 4D-OS128 outperforms two well-studied 4D geometrically-shaped modulation formats: 128SP-16QAM and 7b4D-2A8PSK by up to 0.65dB for bit-interleaved coded modulation at the same spectral efficiency. These gains are experimentally demonstrated in a 11$\times$233Gbit/s wavelength division multiplexing (WDM) transmission system operating at 5.95 bit/4D-sym over 6000km and 9000km for both EDFA-only and hybrid amplification scenarios, respectively. A reach increase of 15% is achieved with respect to 128-ary set-partitioning 16QAM. Furthermore, the proposed 4D-OS128 is also compared to $\boldsymbol{D}_4$ lattice-based constellation and probabilistically-shaped 16QAM with finite blocklength via simulation.