Hybrid Parallelisation of an Algorithmically Differentiated Adjoint Solver

Abstract

We present a novel approach to parallelise an unstructured node-based finite volume solver using a hybrid MPI and OpenMP paradigm. The basic ingredients of this approach are, (i) zero-halo partitioning of the unstructured mesh and (ii) shared node residual accumulation. These two ingredients preclude the need to explicitly exchange the state information across partitions, allowing the computations to run independently in each partition. As a consequence, we retain the original loop structure for the numerical flux kernels, which can be parallelised using OpenMP directives. Due to the hand-assembly of reverse-differentiated routines in our adjoint solver, the adjoint MPI recipes presented in earlier work can not be applied without modifications. We present a modified adjoint MPI treatment for two MPI operations in the context of our hand-assembled nonlinear iterative solver, namely: (i) shared node accumulation and (ii) in-place all-reduce summation. We demonstrate the parallelisation approach on our in-house solver mgopt, which uses the Tapenade AD tool to generate the adjoint code. We show preliminary scalability results for a simple 2d problem.

Publication
ECCOMACS conference
Date