Title: Surface plasmon-phonon-magnon polariton in a topological insulator-antiferromagnetic bilayer structure

Abstract: We present a robust technique for computationally studying surface polariton modes in hybrid materials. We use a semi-classical model that allows us to understand the physics behind the interactions between collective excitations of the hybrid system and develop a scattering and transfer matrix method that imposes the proper boundary conditions to solve Maxwell equations and derive a general equation describing the surface polariton in a heterostructure consisting of N constituent materials. We apply this method to a test structure composed of a topological insulator (TI) and an antiferromagnetic material (AFM) to study the resulting surface Dirac plasmon-phonon-magnon polariton (DPPMP). We find that interactions between the excitations of the two constituents result in the formation of hybridized modes and the emergence of avoided-crossing points in the dispersion relations for the DPPMP. For the specific case of a Bi2Se3 TI material, the polariton branch with low frequency below 2 THz redshifts upon increasing the thickness of TI thin film, which leads to an upper bound on the thickness of the TI layer that will allow an observable signature of strong coupling and the emergence of hybridized states. We also find that the strength of the coupling between the TI and the AFM, which is parameterized by the amplitude of the avoided-crossing splitting between the two polariton branches at the magnon resonance frequency, depends on the magnitude of the magnetic dipole and the line width of the magnon in the AFM material as well as on the Fermi energy of Dirac plasmon in the TI. Finally, we predict that materials with extremely high quality, i.e. low scattering loss rate, are essential to achieve an experimentally-observable strong coupling between a TI and AFM.