Parallel Frame Work for Smoothed Particle Hydrodynamics (SPH) Simulations

Principal Investigator:
Xiangyu Hu

Chair of Aerodynamics and Fluid Mechanics, Technische Universität München

Local Project ID:

HPC Platform used:
SuperMUC and SuperMUC-NG of LRZ

Date published:


As a Lagrangian method, Smoothed Particle Hydrodynamics (SPH) ensures Galilean invariance and conserves mass, momentum and energy inherently. It has been explored and demonstrated for a wide range of applications. Moreover, since SPH solves fluid-dynamics equations on discretized particles carrying specific mass, the space resolution can be adapted which is particularly interesting for compressible fluid dynamics. Due to simple coupling with a gravitational solver, SPH is a preferred approach in computational astrophysics. Other effects, such as radiation fields and magnetic fields, can also be included and help to explore the formation and evolution of galaxies. For incompressible or weakly compressible hydrodynamics, recently there has been an increasing interest to employ multi-resolution SPH to locally resolve fluid details of interest in a more efficient way. Several open-source frameworks exist for the large-scale parallel simulation of particle-based methods in which the resolution of simulation is fixed. Some preliminary work has been published to tackle the difficulties encountered in extending codes with adaptive-resolution capability, e.g. neighbor lists for adaptive resolution, and variable resolution algorithms for particle-based simulations. However, the support for fully parallelized adaptive-resolution in DSM systems is generally still limited in the aforementioned codes.

To deal with adaptive-smoothing-length SPH in high-performance parallel computing, a versatile and flexible framework is required to handle a set of common issues, e.g. domain decomposition, adaptive data structure, parallel fast neighbor search, data communication, etc. A state-of-the-art code with adaptive-resolution capability is the tree-based SPH solver GADGET-2 [4]. GADGET-2 provides a hierarchical representation of particles using the tree method. The domain decomposition is handled based on the Peano–Hilbert curve. A collective hypercube communication strategy is developed to hide communication latency. In the past decades, this method has become increasingly popular for SPH as well as particle-based N-body methods.

Description of the project

The present research project focuses on an alternative approach by introducing a new multi-resolution parallel framework employing several algorithms from previous work. The objective is to seek a succinct yet unified solution to address the existing challenges. In order to integrate the aforementioned algorithms as a consistent and efficient system, several new techniques are developed to overcome the difficulties encountered and to optimize the parallel performance. Furthermore, which this newly development parallel frame work, several applications have explored.

A new multi-resolution parallel framework for SPH

An adaptive rebalancing criterion and monitoring system is developed to integrate the Centroidal Voronoi Particle (CVP) partitioning method as rebalancer to achieve dynamic load balancing of the system. A localized nested hierarchical data structure is developed in cooperation with a tailored parallel fast-neighbor-search algorithm to handle problems with arbitrarily adaptive smoothing-length and to construct ghost buffer particles in remote processors. The concept of “diffused graph” is proposed to improve the performance of the graph-based communication strategy. By utilizing the hybrid parallel model, the framework is able to exploit the full parallel potential of current state-of-the-art clusters based on Distributed Shared Memory (DSM) architectures. A range of gas dynamics benchmarks are investigated to demonstrate the capability of the framework and its unique characteristics. The performance is assessed in detail through intensive numerical experiments at various scales.

A Lagrangian Inertial Centroidal Voronoi Particle method for dynamic load balancing in particle-based simulations

We develop a Lagrangian Inertial Centroidal Voronoi Particle (LICVP) method to extend the original CVP method to dynamic load balancing in particle-based simulations. Two new concepts are proposed to address the additional problems encountered in repartitioning the system. First, a background velocity is introduced to transport Voronoi particles according to the local fluid field, which facilitates data reuse and lower data redistribution cost during rebalancing. Second, in order to handle problems with skew-aligned computational load and large void space, we develop an inertial-based partitioning strategy, where the inertial matrix is utilized to characterize the load distribution, and to confine the motion of Voronoi particles dynamically adapting to the physical simulation. Intensive numerical tests in fluid dynamics simulations reveal that the underlying LICVP method improves the incremental property remarkably without sacrifices on other objectives, i.e. the inter-processor communication is optimized simultaneously, and the repartitioning procedure is highly efficient.

A Consistent Parallel Isotropic Unstructured Mesh Generation Method based on Multi-phase SPH

We propose a consistent parallel unstructured mesh generator based on a multi-phase SPH method. A set of physics-motivated modeling equations are developed to achieve the targets of domain decomposition, communication volume optimization and high-quality unstructured mesh generation simultaneously. A unified density field is defined as the target function or both partitioning the geometry and distributing the mesh-vertexes. A multi-phase SPH method is employed to solve the governing equations. All the optimization targets are achieved implicitly and consistently by the particle relaxation procedure without constructing triangulation/tetrahedralization explicitly. The target of communication reduction is achieved by introducing a surface tension model between distinct partitioning sub-domains, which are characterized by colored SPH particles. The resulting partitioning diagram features physically localized sub-domains and optimized interface communication. The target of optimizing the mesh quality is achieved by introducing a tailored equation-of-state (EOS) and a smooth isotropic kernel function. The mesh quality near the interface of neighboring sub-domains is improved by gradually removing the surface-tension force once a steady state is achieved. The proposed method is developed basing on a new parallel environment for multi-resolution SPH to exploit both coarse- and fine-grained parallelism. A set of benchmarks are conducted to verify that all the optimization targets are achieved consistently within the current framework.

Journal Publications

Zhe J, Fu L, Hu XY, Adams NA (2020) A consistent parallel isotropic unstructured mesh generation method based on multi-phase SPH, Computer Physics Communications 363 (2020) 112881.

Zhe J, Fu L, Hu XY, Adams NA (2019) A Lagrangian Inertial Centroidal Voronoi Particle method for dynamic load balancing in particle-based simulations, Computer Physics Communications 239 (2019) 53–63.

Zhe J, Fu L, Hu XY, Adams NA (2019) A new multi-resolution parallel framework for SPH, Computer Methods in Applied Mechanics and Engineering, 346:1156--1178.

Scientific Contact

Xiangyu Hu
Lehrstuhl für Aerodynamik und Strömungsmechanik
Technische Universität München
Boltzmannstr. 15, D-85748 Garching bei München (Germany)
e-mail: [@]

LRZ project ID: pr53vu

September 2020

Tags: LRZ TUM Computational Fluid Dynamics