Building a bridge to the LUMI supercomputer
Written by Arne Vollertsen
Invisible helper, gatekeeper, and accountant rolled into one: The Puhuri project aims at making it easier for researchers to access and use the new pre-exascale LUMI supercomputer.
Supercomputers are the race cars of research. But while Formula 1 cars are limited to be piloted by a select few, supercomputers should be used as much and by as many researchers as possible, to get the most out of our investment in this expensive machinery.
To achieve that, you need a technical platform making access and usage easy and manageable. This is where Puhuri comes in. Named after the spirit of cold and winter in Finnish mythology Puhuri is building a digital bridge between LUMI and its users.
A valuable resource
Supercomputers are expensive machines, and the new LUMI machine, which will be taken into production in spring 2021, is no exception. Located in Kajaani, Finland, and with a total budget of 207 million euros it is a valuable resource indeed. Furthermore, like other high performance computers, it has a fairly short lifespan - between 4 and 6 years.
So, no wonder the 9 countries participating in the LUMI consortium are eager to achieve the best possible return on their investment. The Puhuri project is designed to do just that, playing multiple roles, serving as invisible helper, gatekeeper, and accountant as well.
European project
ALUMI is part of the EuroHPC Joint Undertaking, a 1 billion euro endeavour to build a European infrastructure of next-generation high-performance computers, consisting of 8 hosting sites, with 5 petascale systems and 3 pre-exascale systems.
The pre-exascale systems are located in Bologna, Italy, Barcelona, Spain, and Kajaani, Finland, and each of the three will be about ten times more powerful than the most powerful supercomputer currently in Europe.
Researchers in many different fields rely on supercomputers. Climate scientists run complex Earth System Models on them to predict long-term effects of climate change. Engineers use supercomputers to optimize the design and structure of airplane wings. Linguists use heavy computation as well, e.g. to develop tools for processing human speech and writing to be recognized by computers, for machine translation and human speech recognition.
In high demand
LUMI’s super powers are in high demand, and not only the 9 LUMI-countries will be using the machine, so will researchers from other countries in Europe and across the world. Furthermore, European industry and SMEs are invited to use LUMI’s superpowers as well.
This means the system needs to be able to handle many different groups of users, enabling them to use LUMI in an easy and secure fashion, and moreover helping them to manage the processing power put at their disposal. Puhuri will develop middleware to bridge the gap between LUMI and various national portals for access and resource allocation.
Gatekeeping
Obviously, gatekeeping is important. You have to be sure that only people authorized to do so can access LUMI. For that you need an identity management solution, enabling users to identify themselves and for the system to authorize their access. Such Authentication and Authorization Infrastructures (AAIs) already exist on national level and for specific research communities. Now Puhuri is building a generic AAI, functioning as a middle layer between LUMI and existing national and community specific portals. This means that, instead of needing a separate password to log in to LUMI, a researcher can access it using his or her home organisation login and password.
Applications and accounting
In addition to gatekeeping, resource allocation management and accounting is equally important. Researchers who want to use the LUMI machine are not just queuing up and waiting in line, first come, first served.
They have to write an application describing their research project and the computing resources they need (GPUs, CPUs and storage), and submit it to the LUMI resource management board. The board then distributes LUMI’s computing resources, based on a peer-review process and on the share of resources assigned to each LUMI country.
Divided into shares
LUMI is divided into “shares”. Half of its resources belong to the EuroHPC Joint Undertaking, the other half belongs to the LUMI Consortium countries Belgium, Czech Republic, Denmark, Estonia, Finland, Norway, Poland, Sweden, and Switzerland. As mentioned, industry and SMEs can apply for resources, and LUMI will also have a channel for urgent computing, related to for instance national security, pandemics and other time-critical tasks.
Currently, the LUMI partners have their own national portals for applying for computing resources. Puhuri will develop middleware that connects them to LUMI, and furthermore provide an interface with an accounting and billing service, showing users for instance what services they have used, how many billing units that remain etc. Again, the aim is to ease access and interaction between users and the computing resources put to their disposal.
Generic solution
Generic – that is one of the key words for what Puhuri wants to achieve in its two-year project run. It is building a generic AAI solution, meaning that it is meant to work not only on LUMI, but on other HPC resources as well. The Puhuri ambition is to develop a digital authorisation and resource allocation platform for accessing HPC resources that can be used by other similar initiatives, instead of them having to invent an ad-hoc solution.
In short, Puhuri - invisible helper, gatekeeper, and accountant rolled into one – is positioning itself as the middleman of the future, when it comes to seamless access to supercomputing for researchers.