Solution:

A large-scale environmental model – the Danish Eulerian Model (DEM) was implemented on the new petascale supercomputer DISCOVERER, installed last year by Atos Company in Sofia Tech Park, Bulgaria. The machine is part of the emerging supercomputing network under development by the European High-Performance Computing Joint Undertaking (EuroHPC JU).

DEM is mathematically represented by a system of partial differential equations for calculating the concentrations of a number of pollutants in the atmosphere above a large geographical region (including the European continent, the Mediterranean and parts of Asia, North Africa and the Atlantic ocean). The main physical and chemical processes (horizontal and vertical wind, diffusion, chemical reactions, emissions and deposition) are adequately represented in the system.

The MPI standard library is used as a main parallelization tool.  MPI parallelization is based on the space domain partitioning. The space domain is divided into sub-domains and each MPI task works on its own sub-domain. On each time-step there is no data dependency between the MPI tasks on both the chemistry and the vertical exchange stages. This is not so on the advection-diffusion stage. Spatial grid partitioning between the MPI tasks requires overlapping of the inner boundaries and exchange of certain boundary values on the neighbour subdomains for proper treatment of the boundary conditions.

The subdomains are usually too large to fit into the fastest cache memory of the corresponding CPU. In order to achieve good data locality, the smaller calculation tasks are grouped in chunks (if appropriate) for more efficient cache utilization. This is done in order to reduce the data transfer between the cache and the main (slower access) memory. The size of chunks should be tuned with respect to the cache size of the target machine.

Scientific impact:  

DEM is a powerful and flexible large scale air pollution model, capable of calculating the levels of concentration for a number of dangerous pollutants and other chemically active species interacting with them (precursors), over a long time period. Moreover, various accumulative quantities (AOT40, AOT60, etc.) can be calculated on yearly basis, which have significant impact in the area of agriculture (on the yield of crops in particular), forestry, wildlife and human health. Over the years it has been successfully applied in many scientific and practical problems in various important areas (environmental, medical, social, economic, etc.).

Benefits: 

The unified parallel version of the Danish Eulerian Model (UNI-DEM) proved to be scalable and use efficiently the computational power of the new petascale EuroHPC supercomputer Discoverer (shown in the table below). It produces a huge amount of output results in quite reasonable time, which can be used in different long-term environmental studies and simulations in various areas:

• Air pollution evaluation and protection measures;
• Human healthcare;
• Economics of agriculture (yield & losses of crops estimation);
• Forestry and wildlife protection;
• Global climate changes consequences simulation;
• Simulation of industrial accidents and natural disasters.