Detailed Simulation of Turbulent Mixed-Mode Combustion Towards Exascale
Jordan A. Denev
Steinbuch Centre for Computing, Karlsruhe Institute of Technology
Local Project ID:
HPC Platform used:
JUWELS of JSC
A large part of the world’s primary energy production is based on combustion. According to the International Energy Outlook, combustion will remain an important energy source for the next few decades. Therefore, in order to reduce greenhouse gas emissions and other pollutants, it is imperative to design cleaner and more efficient combustion devices. This, however, requires to gain a better understanding of the fundamental physics underlying combustion.
In many practical combustion devices, fuel and oxidizer are neither perfectly mixed nor perfectly separated before they react. This may be due to technical limitations or even by design to create a more stable flame. These types of partially premixed flames are challenging to simulate because many models assume that fuel and oxidizer are perfectly mixed or separated. Because of this, a highly resolved simulation of a partially premixed flame of laboratory scale  is conducted which allows to investigate the properties of partially premixed flames in great detail.
The simulations are performed with an in-house code for reacting flows based on OpenFOAM [2,3]. The code has shown excellent scaling behavior with more than 25,000 CPU cores  and includes a number of performance optimizations that allow these kinds of large scale simulations. Most notably, a custom load balancing approach has been implemented which is specifically tailored to reacting flows . Additionally, a converter tool has been developed which creates highly optimized C++ source code for the computation of chemical reaction rates . With these two optimizations, simulation times can be decreased by up to 60 %.
The accuracy of the solver has been validated with many test cases. One such test case is the canonical Taylor-Green vortex. Starting from an initial flow distribution in a box, a pseudo turbulent flow develops over time. This is depicted in Figure 1 by the Q-criterion.
The dissipation rate in the flow field can be evaluated and compared against the solution of a spectral DNS code (Figure 2). The results shows very good agreement between OpenFOAM and the spectral code on the same computational grid.
Simulation and Results
The simulation of the partially premixed flame follows an experimental setup  and is split into two pieces: In a first simulation , solely the mixing of the fuel (methane) and the oxidizer (air) is simulated. Figure 3 shows the computational setup: Methane (colored in red) is provided from a central pipe on the left. Air is supplied from a surrounding pipe, shown in blue. The careful simulation of the mixing process is important to yield the correct behavior of the flame in the subsequent simulation. In total, the simulation of the mixing process is run on a three-dimensional mesh consisting of 300 million hexahedral cells.
Using the results of the mixing process, a subsequent simulation is performed to study the flame. Figure 4 shows how the partially premixed flow of methane and air exits the pipe on the left. The flow is turbulent and therefore creates a complex interaction between vortices in the flow field and the chemical reactions. This is visualized by the iso-surface of vorticity, which illustrates how the flame is affected by the turbulent flow. The flame itself is shown on a 2D cutting plane colored by the temperature field. This simulation considers more than 200 different chemical reactions and is performed on a computational grid with more than 300 million cells. It was run mainly on the Hazel Hen supercomputer at HLRS with additional evaluations on JUWELS at JSC.
Comparison with experimental data have shown very good agreement with the simulation results , which have been compiled into a database. This database is available to help improve existing combustion models and help to create new ones which can simplify the simulation of partially premixed flames.
Thorsten Zirwes1,2, Jordan A. Denev1, Feichi Zhang2, Peter Habisreuther2, Henning Bockhorn2, Dimosthenis Trimis2
1 Steinbuch Centre for Computing, Karlsruhe Institute of Technology
2 Engler-Bunte-Institute, Chair of Combustion Technology, Karlsruhe Institute of Technology
 Barlow, R., Meares, S., Magnotti, G., Cutcher, H., Masri, A.Local extinction and near-field structure in piloted turbulent CH4/air jet flames with inhomogeneous inlets, Combustion and Flame 162 (10), 2015, 3516–3540. doi:10.1016/j
 Zirwes, T., Zhang, F., Habisreuther, P., Hansinger, M., Bockhorn, H., Pfitzner, M., Trimis, D. Quasi-DNS Dataset of a Piloted Flame with Inhomogeneous Inlet Conditions, Flow, Turbulence and Combustion. 2019: doi: 10.1007/s10494-019-00081-5
 Zhang, F., Bonart, H., Zirwes, T., Habisreuther, P., Bockhorn, H., Zarzalis, N. Direct Numerical Simulation of Chemically Reacting Flows with the Public Domain Code OpenFOAM. in: Nagel, W.E., Kröner, D.H., Resch, M.M. High Performance Computing in Science and Engineering ’14. Springer. 2014: 221–236; doi:10.1007/978-3-319-10810-0_16
 Zirwes, T., Zhang, F., Denev, J.A., Habisreuther, P., Bockhorn, H. Automated Code Generation for Maximizing Performance of Detailed Chemistry Calculations in OpenFOAM. in: Nagel, W.E., Kröner, D.H., Resch, M.M. High Performance Computing in Science and Engineering ’17. Springer. 2018: 189–204; doi:10.1007/978-3-319-68394-2_11
 Zirwes, T., Zhang, F., Habisreuther, P., Denev, J.A., Bockhorn, H., Trimis, D. Optimizing Load Balancing of Reacting Flow Solvers in OpenFOAM for High Performance Computing. Proc. of 6th ESI OpenFOAM User Conference. ESI-OpenCFD. 6: 1–13, https://www.esi-group.com/sites/default/files/resource/other/7400/student-abstract_zirwes_karlsruhe-institute-of-technology_optimizing-load-balancing-of-reacting-flow-solvers-in-openfoam-for-high-performance-computing1.pdf
 Zirwes, T., Zhang, F., Denev, J.A., Habisreuther, P., Bockhorn, H., Trimis, D. Improved Vectorization for Efficient Chemistry Computations in OpenFOAM for Large Scale Combustion Simulations. in: Nagel, W.E., Kröner, D.H., Resch, M.M. High Performance Computing in Science and Engineering ’18. Springer. 2018: 209–224; doi:10.1007/978-3-030-13325-2_130
Dr.-Ing. Jordan A. Denev
Karlsruhe Institute of Technology
Steinbuch Centre for Computing
Hermann-von-Helmholtz-Platz 1, D-76344 Eggenstein-Leopoldshafen (Germany)
e-mail: jordan.denev [@] kit.edu
JSC project ID: chka20