Title: A transformer-based neural operator for large-eddy simulation of turbulence

Abstract: Predicting the large-scale dynamics of three-dimensional (3D) turbulence is challenging for machine learning approaches. This paper introduces a transformer-based neural operator (TNO) to achieve precise and efficient predictions in the large-eddy simulation (LES) of 3D turbulence. The performance of the proposed TNO model is systematically tested and compared with LES using classical sub-grid scale (SGS) models, including the dynamic Smagorinsky model (DSM) and the dynamic mixed model (DMM), as well as the original Fourier neural operator (FNO) model, in homogeneous isotropic turbulence (HIT) and free-shear turbulent mixing layer. The numerical simulations comprehensively evaluate the performance of these models on a variety of flow statistics, including the velocity spectrum, the probability density functions (PDFs) of vorticity, the PDFs of velocity increments, the evolution of turbulent kinetic energy, and the iso-surface of the Q-criterion. The results indicate that the accuracy of the TNO model is comparable to the LES with DSM model, and outperforms the FNO model and LES using DMM in HIT. In the free-shear turbulence, the TNO model exhibits superior accuracy compared to other models. Moreover, the TNO model has fewer parameters than the FNO model and enables long-term stable predictions, which the FNO model cannot achieve. The well-trained TNO model is significantly faster than traditional LES with DSM and DMM models, and can be generalized to higher Taylor-Reynolds number cases, indicating its strong potential for 3D nonlinear engineering applications.