Title: Uplink and Downlink MIMO-NOMA with Simultaneous Triangularization

Abstract: In this paper, we consider the uplink and downlink precoder design for two-user power-domain multiple-input multiple-output (MIMO) non-orthogonal multiple access (NOMA) systems. We propose novel uplink and downlink precoding and decoding schemes that lower the decoding complexity at the receiver by decomposing the MIMO-NOMA channels of the users into multiple single-input single-output (SISO)-NOMA channels via simultaneous triangularization (ST) of the MIMO channels of the users and a low-complexity self-interference cancellation at the receiver. The proposed ST MIMO-NOMA schemes avoid channel inversion at transmitter and receiver and take advantage of the null spaces of the MIMO channels of the users, which is beneficial for the ergodic achievable rate performance. We characterize the maximum ergodic achievable rate regions of the proposed uplink and downlink ST MIMO-NOMA schemes, and compare them with respective upper bounds, baseline MIMO-NOMA precoding schemes, and orthogonal multiple access (OMA). Our results illustrate that the proposed schemes significantly outperform the considered baseline MIMO-NOMA precoding schemes and OMA, and have a small gap to the respective upper bounds for most channel conditions and user rates. Moreover, we show that a hybrid scheme, which performs time sharing between the proposed uplink and downlink ST MIMO-NOMA and single-user MIMO, can improve performance even further.