ELEMENTARY PARTICLE PHYSICS

Ultra-Thin Foils (UTFs): Generation of high energy photons by the "slingshot" effect
Researchers: Karl-Ulrich Bamberg, Hartmut Ruhl

A very strong and very short (only a few cycles) – but experimentally available – circularly polarized laser pulse can “press” a major fraction of the electrons out of a 10 nm thick carbon-like foil. The generated electric field strongly accelerates the electrons back to the ions, and relativistic effects create an even shorter AXP useful to “film” protein folding which happens on the time scale of 10e-18 seconds.

Together with AWAKE (see below), the nano-foil project is one of our biggest simulations requiring half a trillion grid cells. Due to this only few output steps are possible. The video below (Fig. 0) shows a 3D-Volume rendering of such a scenario (reduced resolution). During this project, the focus was placed on frequency analysis, which therefore also requires information from every time step. One goal consisted in the extension of the special output routines for better frequency sampling. In addition, we could reduce memory footprint of the simulation from 16 islands of SuperMUC Phase 1 (requiring “Block Operation”) to 8 islands, making the ultra-thin foil simulations much more feasible.

AWAKE: Linear lepton acceleration on the scale of meters instead of kilometers
Researchers: Nils Moschüring, Karl-Ulrich Bamberg, Hartmut Ruhl

The AWAKE project studies the interaction of a 450 GeV proton beam of the SPS pre-accelerator at CERN with a 10 m long plasma. Moving window technology allows for reducing the active memory footprint and the costs by a factor of 30 to about 3% of a full simulation. Nevertheless, every single output dump still takes 3 TB and checkpoints may take even up to 12 TB. For collaboration with experimentalists, these data were still not sufficient, as time-averaged observables requiring information from every single time step were also demanded. Therefore, instead of studying the beam filamentation at higher plasma densities, we redesigned the simulation and implemented heavy inline data processing and analysis in consultation with Dr. Konstantin Lotov (Head of Simulation Efforts of AWAKE).

We also ran a set of simulations with lower plasma density in the linear regime as benchmark with quite convincing results. However, especially the inline data processing on large scales led to a significantly increased amount of collective communication. This revealed an MPI issue most likely related to the IBM-MPI behavior on SuperMUC Phase 1 (Fig. 2).

Exhaustive investigations carried out together with experts from the LRZ Astrolab and engineers from IBM found leads that the problem could be related to the interaction of the collective and point-to-point communication patterns in the PSC with intra-node RDMA transfers. After their elimination the problems did not reappear.

Another big challenge consisted in the fact that the available IBM implementation, “pami_tune” provided only one algorithm for “MPI_GatherV”, which apparently used only “MPI-Root” as receiver. As INTEL did not scale to that many SuperMUC islands (at that time), a custom-tuned tree-like algorithm was written for the PSC, decreasing the wall-clock time from several hundred seconds (on 4-8 islands) to usually less than 40 ms, which is a speed-up factor of more than 2000.

After having resolved these issues, we were able to provide the experimentalists with all the information they needed (Fig. 3) to reach the initial goal of confirming that the reduced codes (fluid-based and 2D cylindrically symmetric) can accurately represent the relevant physics as well as fully kinetic 3D simulations.

The Head of Simulation Efforts of AWAKE confirmed that the respective results showed high concordance, and by now also coincide with initial experimental results, e.g. the PSC simulations correctly showed that there would be no hosing, a result that can only be predicted with fully kinetic 3D simulations. This represents an unprecedented benchmark at this extreme scale for Particle-in-Cell codes. This work has been published as “First Fully Kinetic 3D Simulation of the AWAKE Baseline Scenario” by N. Moschüring, K. Lotov, K. Bamberg, F. Deutschmann, H. Ruhl (2019 [7]).

Mass-Limited Targets (MLTs): Ion acceleration for medical purposes
Researchers: Viktoria Pauw, Karl-Ulrich Bamberg, Hartmut Ruhl

The approach consists in the long/strong acceleration of the electrons and the use of the thereby generated electric field to “pull” and therefore accelerate the protons. This is the direct contrast to AWAKE, where protons accelerate leptons.

Substantial progress was made in finding a parameter range that offers a higher ratio for the conversion of laser pulse energy into fast ion energy when using MLTs levitated in a Paul trap [5] to produce fast ions. In this project we tested different pulse shapes, field strengths and target geometries and densities.

By altering these parameters, the dynamic of the acceleration process can be shifted between different regimes like target normal sheath acceleration (TNSA), Coulomb Explosion (CE) and Radiation Pressure Acceleration (RPA). We studied the transition between these different dynamics and the properties of their fast ion spectrum and found that the maximum proton energies rise approximately linearly with pulse field strength for the near solid targets that were used in the experiments that were run in parallel to the simulation efforts.

Due to the necessary resolution, a typical simulation requires about 20 billion grid cells and runs approximately 12 hours on one island on SuperMUC Phase 1. Larger ones require up to 4 islands.

The simulations revealed that the absorption of laser energy can be enhanced by lowering the density of the target at primary pulse interaction. Experimentally, this can be achieved by pre-expanding the target with a minor pulse before the main interaction. With a target at roughly critical density (n_c), the laser can penetrate the target completely and we have an enhanced RPA effect in addition to the coulomb explosion observed so far in experiments and simulations.

Additionally, the length of the pulse is adjusted to the expansion time of the exploding target. This leads to an increase of maximum proton energy from 100 MeV for solid targets to 400 MeV for less dense targets with a pulse energy of 100 J. If these results can be reproduced in experiment, it will be a significant improvement as compared to the older results, as ion energies of a few 100 MeV are then within reach at much smaller pulse energies that can be delivered by relatively common laser facilities. This set-up also concentrates the accelerated protons much more in the forward direction (Fig. 4/mp4-Video) compared with the Coulomb exploding situation. The work was presented at the GSI-PHEDM conference Hirschegg in January 2020 as "PIC simulation of laser irradiated micro-plasma with varying density" and won the "Laser and Particle Beams" Young Scientist Award. The corresponding paper will soon be published.

Ion Wave Breaking Acceleration (IWBA): Experimentally easy available laser driven ion acceleration in gases and foam
Researchers: Bin Liu, Karl-Ulrich Bamberg, Hartmut Ruhl

Compared to the results of previous IWBA projects, the goal was to simplify the setup for experimentalists by finding laser parameter sets that should be commonly available for experimental laser physicists. The pulse form is now a pure Gaussian, in time as well as in space, and the laser intensity is lower than 10²¹ W/cm² . The difficulty to overcome is then the reduced parameter window for the IWBA regime. For example, with an intensity of 6 ∙ 10²⁰ W/cm², the optimal initial plasma density has to be in the small range of 6.5n_c to 8.5n_c . Up to now, there is no good theory to analytically describe these phenomena. So it is necessary to scan the parameters very carefully to find the small window. Furthermore, the ion trapping happens in a very small region of space, therefore requiring high resolutions of at least 50 cells per micron [2, 3]. Larger runs (Fig. 5) usually require 2000-2560 cores taking 24-48 hours resulting in about 100 kilo-core-hours per run. The parameter scan consisted of several hundred runs (also smaller ones) on SuperMUC and also on its “little sister” Hydra (for “throughput”). The typical output is about two Terabyte per run being reduced to some Gigabytes during post-processing.

QED Vacuum break down in counter propagating circularly polarized laser pulses: Matter-antimatter production
Researchers: Fabian Deutschmann, Constantin Klier, Karl-UIrich Bamberg, Hartmut Ruhl

For very strong laser fields, effects like radiation reaction have to be accounted for. For the PSC a “photon” system was implemented for two reasons: On the one hand, the required resolution to resolve the radiation reaction is way too high even for moderate laser fields. On the other hand, the classic Maxwell equations are insufficient for stronger fields (a₀ ~ 1000). The photons are generated by event generators for both cases, based on Monte-Carlo methods described in the theses of Constantin Klier and Fabian Deutschmann. Furthermore, adaptive time sub-cycle steps were necessary to implement correct dynamics and rates in very strong acceleration cases.

Because of the high memory requirements due to the dynamic production and rebalancing of particles, these simulations needed a complete island of SuperMUC Phase 1. The need for good statistics and the arithmetically complex equations to be solved required more than 120 wall-clock hours runtime, only for the results shown here.