ASTROPHYSICS

Galaxies like our Milky Way are shiny islands of stars with an intricate internal structure that is only partially revealed by optical light. Gas and dust clouds permeate the space between the stars and are constantly stirred up by supernova explosions and radiation outbursts of supermassive black holes in active galactic nuclei. Furthermore, most of the matter in galaxies is contained in a puzzling invisible component, the dark matter, which holds the galaxies together as some kind of cosmic glue. On still larger scales, galaxies are observed to be ordered in a delicately knit “cosmic web”. How these complex entities and their particular spatial arrangement has formed out of the hot matter and radiation soup left behind by the Big Bang is one of the most difficult problems in astrophysics.

In order to gain a theoretical, ab-initio understanding of the structure formation process and the role different physical processes play in shaping galaxies, simulations that account for the multi-scale physics involved are the tool of choice. As a starting point, such calculations rely on initial conditions that are known and well constrained by modern observational data. The calculations then need to accurately compute the dominant physical force, namely gravity, acting upon all matter, within the accelerating expansion of the Universe. Furthermore, (magneto) hydrodynamical processes for modeling the evolution of the gaseous component of the Universe need to be followed. Finally, one must account for all other relevant astrophysical processes: from the atomic level interactions that govern radiative cooling of a metal-enriched gas, to the formation of stars and supermassive black holes, with their subsequent expulsion of mass, metals, and “feedback” energy, which can impact scales even larger than entire galactic halos.

The Next Generation Illustris Simulations (IllustrisTNG) are a new bold attempt to carry out such calculations. They build up on the success of the “Illustris”-project from 2014 (which had been carried out with GCS and PRACE resources), but improve it in critical ways. Most importantly, IllustrisTNG uses a substantially improved physical and numerical model, and pushes the calculations to much larger sizes. In particular, the new simulations account for a novel kinetic wind model to represent black hole feedback physics, they incorporate magnetic field amplification from high redshift to the present, and they employ new techniques for tracking the production of heavy elements in supernova explosions of different type. The numerical approach is based on the AREPO code, which uses a sophisticated moving-mesh algorithm for hydrodynamics that yields continuous adaptivity and very low advection errors even in highly supersonic parts of the flows.

The ambitious simulation program carried out in IllustrisTNG consists in total of three complementary cosmological simulations. Two of the main runs, TNG300 (a large volume simulation with a periodic boxsize of 300 Mpc), and TNG100 (intermediate volume, boxsize 110 Mpc) were carried out in the GCS-ILLU project. A third, still more difficult calculation, TNG50 (small volume, boxsize 50 Mpc), is the target of a follow-up project. To assess resolution convergence, the TNG300 simulation (as well as the other flagship runs) has been supplemented by other lower resolution simulations, spaced by factors of 8 in mass resolution. And to isolate the effects of the galaxy formation model onto the underlying dark matter distribution, each full baryonic physics run has also been repeated with the same initial condition phases, but including only dark matter and gravity.

To give an idea about the scope of the simulations, the TNG300 simulation was carried out using 24000 processor cores on Hazel Hen and contained 31.25 billion resolution elements, half in the form of dark matter particles and half as Voronoi gas cells. In addition, more than 15 billion Monte Carlo tracer particles for tracking gas flows were included. The simulation required about 100 TB of RAM to execute, corresponding to nearly all of the RAM of the assigned compute nodes. The cumulative data volume of the restart files written and read again in order to carry out the calculation over the coarse of several months was in excess of 10 Petabytes, corresponding to a time-averaged I/O rate of about ~1 GB/sec. The data volume kept permanently for scientific analysis is of the oder of 250 TeraByte for this simulation.

The first results of the IllustrisTNG project demonstrated a spectacular breadth of new theoretical predictions. For example, for the first time these hydrodynamical simulations of galaxy formation have reached sufficient volume to allow precision predictions for clustering on cosmologically relevant scales. In fact, the new IllustrisTNG simulations have determined the non-linear correlation functions and power spectra of baryons, dark matter, galaxies and haloes over an exceptionally large range of scales. This showed that baryonic effects increase the clustering of dark matter on small scales, and damp the total matter power spectrum on scales up to 3 Mpc. The two-point correlation function of the simulated galaxies has been found to agree well with observational data from the Sloan Digital Sky Survey, both as a function of stellar mass and when split according to galaxy color.

Given this agreement, the TNG simulations can make valuable theoretical predictions for the so-called clustering bias of different galaxy samples, which has important cosmological implications. For example, the simulations demonstrate the existence of significant scale dependencies in the bias of different observational tracers of large-scale structure, extending well into the range of the baryonic acoustic oscillations that can be used to reconstruct the expansion history of the universe and thus constrain dark energy. The resulting nominal shifts of the acoustic peaks need to be corrected for in future data analysis with procedures that can be tested with the IllustrisTNG simulations.

Another interesting aspect are the magnetic fields that, for the first time, could be tracked in IllustrisTNG. Magnetic fields are prevalent in galaxies, and their build-up is closely linked to structure formation. The TNG simulations show that structure formation amplifies the tiny initial seed fields to the values observed in low-redshift galaxies (1 − 10 μG). The magnetic field topology is closely connected to galaxy morphology such that irregular fields are hosted by early-type galaxies, while large-scale, ordered fields are present in disc galaxies. This for example allows predictions of the diffuse radio emission of galaxy clusters, and observational forecasts for forthcoming radio telescopes such as the Square Kilometer Array (SKA).

The scientific analysis of the TNG300 and TNG100 simulations is now well under way, and will keep all participating scientists busy for many years to come. It is also planned to make all the data publicly available by the end of 2018, which will further boost the utility of the project for the international astronomical research community. Already now IllustrisTNG exemplifies the indispensable role that high-performance supercomputing plays in modern astrophysics.