Artificial Intelligence and Machine Learning

AI systems that recognise images or translate text are built from neural networks. How such a network is put together has a big influence on how well it works. Finding a good structure usually means building and training many versions, which takes a lot of computing time. Researchers in the group of Prof. Volker Tresp at LMU Munich and the Munich Center for Machine Learning (MCML) developed DiffNAS, a method that uses generative AI to suggest network structures and quick tests to sort out weak ones early. On a standard benchmark, DiffNAS found the best available structures with very little search time. The experiments used more than 100,000 node hours on the Hawk supercomputer at HLRS and additional time on its successor Hunter.

How should an AI model’s training change when it is given more data? Researchers at Forschungszentrum Jülich used JUWELS Booster to investigate how training settings and neural network building blocks affect performance at scale. The BeSL (Better models with Scaling Laws) project established practical relationships between data budgets, learning rates, and batch sizes. It developed a stable way to train networks with learnable activation functions. The results offer guidance for making better use of supercomputing resources when developing AI models.

Deep learning models power everything from image recognition to language translation, but their performance depends heavily on small internal components called activation functions. Researchers at the University of Freiburg have developed a new activation function called GoLU, the Gompertz Linear Unit, that consistently outperforms existing alternatives across a wide range of tasks. Using high performance computing clusters, the team conducted over 112,000 GPU hours of experiments, demonstrating that this deceptively simple mathematical function can make neural networks more accurate and robust.

Since the release of ChatGPT in December 2022, artificial intelligence (AI) has garnered unprecedented media attention. Mainstream media outlets, such as Tagesschau, now regularly report AI-related news, and public discourse on the subject has permeated schools and universities. This widespread attention is the result of significant developments in Deep Learning (DL), which has proven incredibly successful at solving previously unsolvable tasks and outperforming known solutions to established problems. Deep Neural Networks (DNNs) are very powerful, but designing their architectures and training pipelines for novel tasks has traditionally required extensive trial-and-error by human experts. Thus, DNNs have been hard to adopt in new fields.

The SunGANBoost project developed advanced deep learning models capable of generating realistic synthetic solar images from multi-wavelength observations. By training large Generative Adversarial, Autoencoder and Diffusion models on data from NASA’s Solar Dynamics Observatory using the JUWELS Booster GPU supercomputer, the team achieved unprecedented fidelity in reproducing solar structures across wavelengths. The resulting models will support future solar physics research, helping scientists better understand and predict solar activity and its effects on space weather.

Autonomous vehicles must predict the motion of other road users within fractions of a second in complex urban environments with hundreds of lanes, traffic lights, and vehicles. RedMotion addresses this challenge through a novel transformer architecture that learns augmentation-invariant and redundancy-reduced descriptors of road environments. By compressing up to 1,200 local environmental features into exactly 100 compact tokens through self-supervised learning, RedMotion achieves efficient and accurate motion prediction. Training on millions of traffic scenes from the Waymo and Argoverse datasets required extensive parallel computations on the GPU nodes of JUWELS Booster.

Over the last few decades, ab-initio methods such as density-functional theory have become sufficiently accurate to allow for the prediction of many properties of new crystal structures. However, these predictions come at a significant cost, and due to the vastness of the space of possible materials, theoretical material discovery remains one of the most challenging questions in materials science. Machine learning methods, trained on existing databases of ab-initio calculations, have the potential to massively accelerate the process of theoretical materials discovery. One of the most important properties targeted is thermodynamic stability, which is used as a proxy to estimate the probability that a given compound can be synthesized.

Experimental advancements within the last two decades have enabled unprecedented control of quantum systems, posing outstanding challenges for their theoretical description. Our project is based on a novel computational strategy at the intersection of machine learning and quantum physics, utilizing artificial neural networks to efficiently represent quantum wave functions. By leveraging supercomputing resources from FZ Jülich and the Gauss Centre for Supercomputing, we have advanced the theoretical understanding of strongly interacting systems in two dimensions, including the first demonstration of the quantum Kibble-Zurek mechanism.

A team of researchers led by TU Ilmenau Professor Jörg Schumacher have been using the JUWELS supercomputer at the Jülich Supercomputing Centre to run highly detailed direct numerical simulations (DNS) of turbulent flows at the so-called mesoscale—the intermediate range where both small-scale turbulent fluid interactions and large-scale fluid dynamics converge.

A group of researchers from the Fritz Haber Institute and Aarhus University in Denmark have leveraged the power of the JUWELS supercomputer at the Jülich Supercomputing Centre (JSC) to develop a machine learning algorithm that helps predict how specific molecules bind to the surface of a catalyst. Catalysts play an essential role in many chemical processes, and how specific molecules interact with these materials can influence the efficiency, effectiveness, and safety of chemical reactions at an industrial scale.

Impressive progress has recently been made in machine learning where learning capabilities at (super-)human level can now be produced in non-spiking artificial neural networks. A critical challenge for machine learning is the large number of samples required for training. This project investigated new high-throughput methods across various domains for biologically based spiking neuronal networks. Sub-projects explored tools and learning algorithms to study and enhance learning performance in biological neural networks and to equip variants of data driven models with fast learning capabilities. Applications of these learning techniques in neuromorphic hardware and design for their future application in neurorobotics were also included.

The need to reduce the skin-friction drag of aerodynamic vehicles is of paramount importance. Nominally 50% of the total energy consumption of an aircraft or high-speed train is due to skin-friction drag. Reducing skin-friction drag reduces fuel consumption and transport emissions, leading to vast economic savings and wider health and environmental benefits. In this project, wall-normal blowing is combined with a Bayesian Optimisation framework in order to find the optimal parameters to generate net energy savings over a turbulent boundary layer. It is found that wall-normal blowing with amplitudes of less than 1% of the freestream velocity of the boundary layer can generate a drag reduction of up to 80% with up to 5% of energy saving.