Max Planck Institute for Gravitational Physics, Potsdam (Germany)
Local Project ID:
HPC Platform used:
SuperMUC of LRZ
On the 17th of August 2017, the observation of gravitational and electromagnetic radiation from a binary neutron star coalescence initiated a new era of multi-messenger astronomy . For the first time the coincident detections of short gamma ray bursts, a kilonova, and a gravitational wave (GW) signal connected several high-energy astrophysics phenomena with the collision of the most extreme and unknown stars in the Universe.
While this achievement is already an important scientific breakthrough, one expects multiple observations of merging neutron stars in the next years due to the increasing sensitivity of advanced GW detectors.
To interpret the observations, theoretical studies of binary neutron star systems are necessary. Because of the complexity of the non-linear Einstein Equations coupled to the equations of general relativistic hydrodynamics, numerical relativity simulations are required to describe the system in the last stages of the binary coalescence.
Numerical relativity simulations are a multi-scale and multi-physics problem that requires the solution of nonlinear partial differential equations in complex geometries.
Over the last years, our group has developed numerical methods and codes to perform such simulations to allow predictions of the gravitational-wave and electromagnetic radiation emitted by compact binaries. Aspects we are focusing on are the dynamical interaction between supranuclear-density and the production of accurate gravitational waveforms for a variety of binary parameters.
Dynamical simulations are performed with the BAM code. BAM combines state-of-art methods to deal with black hole spacetimes and shock capturing methods for general relativistic hydrodynamics simulations.
The code is based on the method of lines and uses high-order finite difference stencils for the spatial discretization of the geometric variables, while high resolution shock capturing methods are used for the hydrodynamic variables. The time integration is done with an explicit Runge-Kutta method. The BAM infrastructure also supplies adaptive mesh refinement by a combination of fixed and moving boxes, as well as cubed spheres. The code is written in C and is hybrid OpenMP/MPI parallelized.
It is important to point out that scientific statements can only be made with a bundle of numerical simulations and that individual simulations of a single physical setup using one resolution are almost meaningless. Therefore, we are forced to simulate physical setups with different resolutions to show consistency, to check convergence, and to give proper error bars for the observables. Additionally, we have to span a reasonable range in the parameter space to study the imprint of the binary parameters, as spin, equation of state, total mass, and mass-ratio. Depending on the resolution and parameters considered every individual simulation runs on a few hundred to a few thousand processors. Currently, we have consumed ~100 million CPU hours on SuperMUC within the project pr48pu. We produced ~200 million files and used a maximum of ~110 TB of storage.
With the help of the computational resources granted through the project pr48pu, we have written in the last 2 years 10 peer reviewed articles. Some of the research highlights will be discussed in the following.
Waveform model development:
The main target of gravitational wave astronomy is to extract the properties of the observed system like the stars’ masses or spins from the detected signal. For this purpose the signal is cross-correlated with waveform templates. Therefore, a key to the source identification is the availability of state-of-the-art models of the gravitational-wave signal. Recently, we constructed an analytical closed-form gravitational wave model which employs directly high-resolution and error-controlled numerical relativity data . The latter have been combined with analytical expressions based on post-Newtonian theory, describing the early inspiral when the two stars are still far apart, and on waveforms obtained in the so-called effective-one-body approach . This allowed us to build waveform approximants that are valid from the low frequencies to the strong-field regime and up to merger. Our work  provided for the first time simple, flexible, and accurate models used directly in the data analysis of the first binary neutron star event observed by LIGO and Virgo .
Binary Neutron Star Parameter Space Coverage:
Currently, our collaboration is about to release the first catalog of binary neutron star waveforms with a total of 346 simulations. An important aspect of our work is the use of initial data which fulfill the Einstein Constraint Equations and the equations governing the evolution of the matter variables. These consistent initial data allow for highly accurate predictions of the binary evolution. Furthermore, with the methods presented in  we are able to access large regions of the binary neutron star parameter space.
In particular, we have been the first who performed simulations for spinning neutron stars with a realistic description of the intrinsic rotation of the stars. We have been the first who simulated precessing binary neutron star mergers, i.e., systems in which the orbital plane precesses due to the fact that the spins of the neutron stars are not aligned with the orbital angular momentum. We managed to simulate systems with large mass ratios, in particular mass ratios between 1.5 and 2. Since, although the observed neutron star binaries have currently mass ratios below 1.3, one expects that also systems with higher mass ratios exist, consequently we need to be prepared for upcoming gravitational wave observations in different regions of the parameter space.
Very recently we started the investigation of highly eccentric binary neutron star systems. These systems which can form in globular clusters allow to constrain the Equation of State of neutron star matter by density oscillations induced into the stars during close encounters in the inspiral.
In the future we plan to extend our work on binary neutron star systems and focus on the development of a new pseudospectral code, BAMPS. BAMPS includes already routines for general relativistic hydrodynamics within the framework of discontinous Galerkin methods and will be the next-generation successor to BAM.
S. Bernuzzi, B. Brügmann, S. V. Chaurasia, K. Clough, R. Dudi, F. Fabbri, N. Madrigal, N. Ortiz, D. Radice, W. Tichy, M. Ujevic, F. Zappa
Max Planck Institute for Gravitational Physics Potsdam, University of Parma, University of Jena, Federal University of ABC Sao Paulo, Florida Atlantic University
References and Links
 T. Dietrich, S. Bernuzzi, and W. Tichy, 2017. Closed-form tidal approximants for binary neutron star gravitational waveforms constructed from high-resolution numerical relativity simulations. Phys. Rev. D.96, 12, 121501.
 S. Bernuzzi, A. Nagar, T. Dietrich, and T. Damour, 2014. Modeling the Dynamics of Tidally Interacting Binary Neutron Stars up to the Merger. Phys.Rev.Lett. 114, (2015) 16, 161103.
 T. Dietrich, et al., 2015. Binary Neutron Stars with Generic Spin, Eccentricity, Mass ratio, and Compactness - Quasi-equilibrium Sequences and First Evolutions. Phys. Rev. D.92, 12, 124007
now: Nikhef (National Institute for Subatomic Physics)
Science Park 105
NL-1098 XG Amsterdam, The Netherlands
e-mail: t.dietrich [@] nikhef.nl
NOTE: This report was first published in the book "High Performance Computing in Science and Engineering – Garching/Munich 2018":
LRZ Project ID: pr48pu