Title: A transformer-based neural operator for large-eddy simulation of turbulence

Abstract: Predicting the large-scale dynamics of three-dimensional (3D) turbulence is challenging for machine learning approaches. This paper introduces a transformer-based neural operator (TNO) to achieve precise and efficient predictions in the large-eddy simulation (LES) of 3D turbulence. The performance of the proposed TNO model is systematically tested and compared with classical sub-grid scale (SGS) models, including the dynamic Smagorinsky model (DSM) and the dynamic mixed model(DMM), as well as the original Fourier neural operator (FNO) model, in homogeneous isotropic turbulence (HIT) and free-shear turbulent mixing layer. The numerical simulations comprehensively evaluate the performance of these models on a variety of flow statistics, including the velocity spectrum, the probability density functions (PDFs) of vorticity, the PDFs of velocity increments, the evolution of turbulent kinetic energy, and the iso-surface of the Q-criterion. The results demonstrate that the TNO model exhibits better accuracy than the DSM, DMM, and FNO models in both HIT and the turbulent mixing layer. Moreover, the TNO model has fewer parameters than the FNO model and enables long-term stable predictions, which the FNO model cannot achieve. Besides, the proposed TNO model is much faster than traditional LES with DSM and DMM models, showing great potential in tackling 3D nonlinear engineering problems.