Exascale Computing ALgorithms & Infrastructures Benefiting UK Research (ExCALIBUR)

To continue to make scientific advances on some of the most challenging physical problems facing the world today, it is essential that the UK fully harnesses the power of the world’s most powerful supercomputers as we move into the exascale era and beyond. However, this cannot be achieved without appropriate software: existing simulation codes will not perform effectively on the next generation of supercomputers. The ExCALIBUR programme was set up in 2019 to address this challenge by redesigning high priority computer codes and algorithms. Running until 2025, around 50 projects have been funded across the UK tackling some of the key challenges associated with computing at the exascale.

Knowledge exchange is a major part of ExCALIBUR, and EPCC has taken on the role of coordinating knowledge exchange across the programme. Focussing on ensuring that developments and research findings are shared not only between ExCALIBUR projects, but also connect with and compliment exascale activities going on across the world.

Beyond this, EPCC has been involved in a variety of projects which have been funded by the programme to explore a range of topics. 

Cross Domain Specific Languages (xDSL)

A collaboration between the University of Edinburgh (the School of Informatics and EPCC), Cambridge University, and Imperial College London, the xDSL project aims to revolutionise Domain Specific Languages (DSLs) by developing a common ecosystem built upon MLR. Currently DSL implementations are highly siloed which often results in significant development efforts, a lack of third party tooling, immature technology stacks, and uncertain long term support. Focussing on the PSyclone and Devito DSLs in this project, thanks to our efforts these technologies now leverage a common DSL ecosystem which benefits them both and is reported by a recent ASPLOS paper. 

Furthermore, we have been able to develop a prototype approach where the domain specific abstractions can feedback into general purpose codes, in our case stencil abstractions being applied to Fortran codes via Flang, which delivers significant benefits. An important by-product of this project has been to develop a Python compiler toolkit for MLIR, significantly lowering the barrier to entry in using MLIR and enabling fast prototyping. Indeed, an important success has been the development of an MPI dialect, first in our xDSL tool and which has now been merged into main MLIR, bringing distributed memory abstractions into MLIR for the first time. 

See https://xdsl.dev for more details.

RISC-V testbed

An important question is around the hardware technologies that we can leverage in future exascale supercomputers, which is especially important as we move towards greener supercomputing. RISC-V is an open, community driven, Instruction Set Architecture (ISA) and with around 13 billion RISC-V devices is enjoying very significant growth and popularity. However, RISC-V is yet to gain popularity for HPC workloads and to this end ExCALIBUR have funded a RISC-V testbed which enables scientific programmers to gain free access to RISC-V so that they can experiment with the technology for their workloads. 

As part of this project, we undertook the first independent benchmarking of a large core count high performance RISC-V CPU. This is detailed here, were we ran benchmarks on the 64-core SG2042. An important aim of this work has been to drive increased adoption of RISC-V in HPC, and to this end we have led organisation of the RISC-V for HPC workshop series at ISC and SC which has proven extremely popular.

ExCALIBURML

ExCALIBURML is a three-year collaboration between EPCC and the School of Mathematics at the University of Edinburgh with funding of over £576,000. The aim of the project is the development of an adaptor interface SiMLInt (Simulation and Machine Learning Integration) to allow the outsourcing of suitable, computationally expensive parts of simulation codes, traditionally written in C/C++ or Fortran, and replacing them by approximations from trained ML networks.

ELEMENT

ELEMENT addresses the high priority use case of meshing for the Exascale (ensuring that meshes are of sufficient quality to represent Exascale problems and can be partitioned efficiently to minimise load imbalance) as well as meshing at the Exascale (creating highly scalable solutions able to exploit extreme levels of parallelism). The project is led by EPCC. 
ELEMENT project website

FPGA testbed

Field Programmable Gate Arrays (FPGAs) are configurable, at the electronics level, to directly represent applications. Whilst they have been around for over 40 years, and are ubiquitous in some fields, they are yet to gain acceptance in HPC. However, recent years have seen phenomenal advances in both the capability of the hardware and the software ecosystem, making them a more realistic proposition for HPC codes than ever before. Furthermore, FPGAs are very energy efficient which is crusical as the HPC community looks to embrace Net Zero.

EPCC hosts an FPGA testbed which has been funded by ExCALIBUR. The system is available for developers to experiment with this technology for their workloads to enable them to understand the role that these technologies might play in future exascale machines. More details can be found at the FPGA testbed website

PAX-HPC: Particles at Exascale

PAX-HPC combines skills in Massively Parallel Particle Hydrodynamics (MPPH) and Materials & Molecular Modelling (MMM), including many community partners such as the Materials Chemistry Consortium (MCC) and UK Car-Parrinello Consortium (UKCP). It focuses on particle-based simulations across many length- and time- scales, from atoms to galaxies. The project is led by University College London with EPCC among the co-investigators. 

https://excalibur.ac.uk/projects/pax-hpc-particles-at-exascale/

EXALAT: Lattice Field Theory at the Exascale Frontier

EXALAT is a collaboration led by the University of Edinburgh, with total funding of £365k to build on the recognised expertise in the Lattice Field Theory (LFT) community to use high-end computing to inform and upskill the wider scientific ecosystem for Exascale computing. 
Exalat website