Title: Effect of extended confinement on the structure of edge channels in the quantum anomalous Hall effect

Abstract: Quantum anomalous Hall (QAH) effect in the films with nontrivial band structure accompanies the ferromagnetic transition in the system of magnetic dopants. Experimentally, the QAH transition manifests itself as a jump in the dependence of longitudinal resistivity on a weak external magnetic field. Microscopically, this jump originates from the emergence of a chiral edge mode on one side of the ferromagnetic transition. We study analytically the effect of an extended confinement on the structure of the edge modes. We employ the simplest model of the extended confinement in the form of potential step next to the hard wall. It is shown that, unlike the conventional quantum Hall effect, where all edge channels are chiral, in QAH effect, a complex structure of the boundary leads to nonchiral edge modes which are present on both sides of the ferromagnetic transition. Wave functions of nonchiral modes are different above and below the transition: on the "topological" side, where the chiral edge mode is supported, nonchiral modes are "repelled" from the boundary, i.e. they are much less localized than on the "trivial" side. Thus, the disorder-induced scattering into these modes will boost the extension of the chiral edge mode. The prime experimental manifestation of nonchiral modes is that, by contributing to longitudinal resistance, they smear the QAH transition.