Title: Surface Majorana Flat Bands in $j=\frac{3}{2}$ Superconductors with Singlet-Quintet Mixing

Abstract: Recent experiments have revealed the evidence of nodal-line superconductivity in half-Heusler superconductors, e.g. YPtBi. Theories have suggested the topological nature of such nodal-line superconductivity and proposed the existence of surface Majorana flat bands on the (111) surface of half-Heusler superconductors. Due to the divergent density of states of the surface Majorana flat bands, the surface order parameter and the surface impurity play essential roles in determining the surface properties. In this work, we studied the effect of the surface order parameter and the surface impurity on the surface Majorana flat bands of half-Heusler superconductors based on the Luttinger model. To be specific, we consider the topological nodal-line superconducting phase induced by the singlet-quintet pairing mixing, classify all the possible translationally invariant order parameters for the surface states according to irreducible representations of $C_{3v}$ point group, and demonstrate that any energetically favorable order parameter needs to break time-reversal symmetry. We further discuss the energy splitting in the energy spectrum of surface Majorana flat bands induced by different order parameters and non-magnetic or magnetic impurities. We proposed that the splitting in the energy spectrum can serve as the fingerprint of the pairing symmetry and mean-field order parameters. Our theoretical prediction can be examined in the future scanning tunneling microscopy experiments.