LRZ Welcomes Terrabyte Joint Data Analysis Platform
Press Release –

The high-performance data analytics platform terrabyte, which is jointly run by the DLR and the LRZ, is now fully operational.

A world settlement footprint of Hong Kong. As part of the terrabyte project, data like this will be analyzed by German researchers in conjunction with AI methods. Image credit: DLR

Garching, 15 June 2023Scientists and researchers can now access a huge treasure trove of data packed with information about the state of the Earth: Today, the German Aerospace Center (DLR) and the Leibniz Supercomputing Centre (LRZ) announce the launch of terrabyte, a high-performance data analytics platform (HPDA) that enables researchers in Bavaria and Germany to use satellite and earth observation data for classical simulations and evaluate it using artificial intelligence (AI) methods. The innovative platform is funded by the Bavarian State Ministry of Science and the Arts and the Federal Ministry of Economics and Climate Action.

terrabyte combines classic processors (CPU) with accelerators and graphics processing units (GPU). The core of the platform consists of ThinkSystem SD650-N V2 servers and DSS-G memory from Lenovo. For classic modelling and simulation as well as AI workloads the platform is equipped with 61 CPU nodes, each with two 40-core Intel Xeon Platinum processors, as well as 15 GPU nodes, each featuring four A100 accelerators from NVIDIA. Thanks to Infiniband HDR, terrabyte can process data at a speed of 320GB/s. The data is transferred from the DLR's German Satellite Data Archive in Oberpfaffenhofen to the terrabyte platform at the LRZ in Garching near Munich via a direct network connection running at 100 gigabits per second. This equipment allows the HPDA platform to achieve a performance of up to 1.3 petaflops, which means it can handle up to 1.3 quadrillion floating point operations per second. By autumn this year, terrabyte's performance will be more than doubled. A hot-water cooling system ensures an energy-efficient operation.

As easy to use as cloud services

Similar to the LRZ Data Science Storage (DSS), the platform features around 50 petabytes of additional online storage. The LRZ is responsible for integrating not only the operating system on terrabyte, but also the software stack for its high-performance computers, as well as frequently used, freely available applications for high-performance computing. Users will also find open-source software, tools and algorithms developed by the Earth Observation Center (EOC) and other DLR institutes. These can be used to process a wide range of geoinformation data or to train machine learning applications. Researchers can also implement and use their own algorithms.

DLR users and scientists in Bavaria can access terrabyte online using their LRZ ID. They will be able to manage their data sets and calculations in the same convenient way as offered by popular cloud services. A large part of the satellite data is pre-processed (known as Analysis Ready Data, ARD) and can be used in pre-installed applications. This enables terrabyte to perform a variety of geoscientific analyses that were previously only possible on commercial platforms  due to the large amounts of data and the need for high computing power.

How data helps improve planning and disaster prevention

The DLR’s data treasure trove includes historical information from 50 years of earth observation as well as current radar and multispectral images that satellites send to earth every day. They form the basis for research projects such as the DLR's "World Settlement Footprint", which has tracked in 3D the development of settlements all over the world since 1985, particularly in metropolises and megacities such as Cairo, Mumbai or Rio de Janeiro, and has provided support for urban and transport planners. The data sets of all calculations since 2015 have become industry standard and are used by the United Nations, the World Bank and many other institutions.

Satellite data can now also be used to evaluate locations for offshore wind farms both on a regional and global scale, and to analyse their operation. In this way, it is possible to better assess the contribution of these plants to the national power supply of a country. Earth observation data are also of particular importance when it comes to disaster control: today, satellites provide information on the current state of forests, for example with regard to drought levels. This information can be used to prepare warnings and plan measures against forest fires.

The DLR invested around 19 million euros of German federal funds in the construction and expansion of terrabyte. The LRZ ensures the operation of the HPDA platform, it handles the ongoing operating costs and provides staff. Funding has been provided by the Bavarian Ministry of Science and the Arts.

For the full text of the release, please visit the LRZ website