Building a cloud for Nordic researchers

Researchers across the Nordic region are starting to enjoy a new “elastic” cloud computing infrastructure from where they can collaborate and share systems and resources. The Glenna project has already supported the development of both a Norwegian and a Danish national cloud service into true Nordic services, and more services are in the pipeline.

Cloud computing – the delivery of hosted computer resources and data instead of building and maintaining computing infrastructures in-house – has the same advantages for researchers as for the man on the street, and then some. It allows research institutions to reduce or avoid infrastructure costs, and enables them to focus more on their core activities instead of spending time and money on computer infrastructure. Cloud computing also makes it possible to adjust the use of resources in line with changing or fluctuating demands. “But cloud computing can have pros and cons,” says Project Manager Dan Still from the CSC – IT Center for Science in Finland. He led the aptly named Glenna project from the start in 2014 to the finish in 2016, and is now preparing for Glenna 2, which will continue and expand the work done in Glenna. The name of the projects is taken from an old Icelandic word that can mean “Opening in the clouds,” which is exactly what Still and colleagues have been working on: They are opening cloud computing for specialised services to researchers in the five Nordic countries.

Using the cloud without knowing it

“The concept of cloud computing may seem mysterious to some people, when they are in fact using these services without even knowing that they are in the cloud. A familiar example of a cloud service is Gmail, which gives users the option to log into a web server and read their emails there, and store them, instead of downloading to their computer. In practice, most of Google’s services can be considered cloud computing,” Still explains.

The list of cloud computing service providers has been growing rapidly over the past several years. One famous case comes from the company that became the world’s largest online bookstore shortly after the launch in 2004: They are now not only selling books any more, but are also offering virtual computers on which customers can run their own computer applications via the Amazon Elastic Compute Cloud. The Windows OneDrive service comes pre-installed on computers with Windows 10. There are many other similar examples to be found.

One of the risks that comes with buying cloud computing services from commercial companies is that they can go bust or be sold to new owners that the customers do not trust. There are also important questions about the ownership of data stored on private servers, and the services may be more susceptible to hacking. Moreover, commercial services can be quite expensive and not always close to the data.

Sharing knowledge, establishing best practices

Researchers are typically more concerned about data location and security than the average cloud user. That is why NordForsk and the Nordic e-infrastructure Collaboration (NeIC) decided to launch the Glenna Nordic Cloud project. The purpose was to share knowledge and set best practices on managing cloud services. The goal was in a similar vein: To create a number of shared Nordic cloud services, driven by the needs of the Nordic researchers.

Rather than creating entirely new infrastructures from scratch and deal with cost and legacy issues, NeIC decided to take advantage of the existing infrastructure and allow for future flexibility.

“Let’s say that the ideal service that a Norwegian researcher needs is located in Sweden. Wouldn’t it be nice if that researcher could log in to the Swedish system and use the service from Norway? It is more cost-efficient to develop existing national cloud services so that they can be used in all the Nordic countries, than to develop separate national services for each country. At the same time, we are leveraging knowledge that is already available,” Dan Still explains.

Cloud storage and portal services

The Glenna project has already completed the expansion of one Danish and one Norwegian service into Nordic services. The Danish DeIC data portal now provides cloud storage for researchers in all the five Nordic countries, and can be compared to a “Dropbox for researchers” with additional features. The Norwegian Lifeportal service is now easing access to high performance life science computing resources for researchers in all the Nordic countries, from an original starting point of having been developed for Norwegian researchers only. Both services can be accessed through single sign-on by users belonging to the national identity federations in the respective Nordic countries.

These examples illustrate one of the basic ideas behind Glenna: To exchange services that have already been created in one Nordic country. Development of both the DeIC data service and the Lifeportal as national services was costly, so it made good economic sense to expand them into Nordic services for a relatively small additional investment.

“The DeIC data portal and the Lifeportal service are now accessible cross-border, and a nice feature is that Nordic researchers don’t have to apply or register themselves as users. They can access the services if they are registered as students or researchers in one of the Nordic countries,” says Still.

In addition, Glenna has supported the meteorological institutes in Finland and Norway in developing a weather forecasting model called Harmonie MUSC into a cloud-based programme.

The Glenna project also succeeded in making the Nordic National clouds elastic, which means that the systems are able to adapt to workload changes by provisioning and de-provisioning resources in an autonomic manner, such that at each point in time the available resources match the current demand as closely as possible at each point in time.

“The Nordic countries share a high level of trust amongst themselves, and this is a prerequisite for the implementation of Nordic solutions instead of national ones,” Still adds.

Weather forecasting moves into the cloud

”The most important thing is that this was a learning exercise and a successful proof-of-concept. We learnt that it can be surprisingly easy to install heavy weather forecasting programs in a cloud environment”, says researcher Niko Sokka at the Finnish Meteorological Institute.

Modern weather forecasting relies heavily on measurements of atmospheric factors like temperatures, humidity, wind speeds and so on, which are entered into massive computer models designed to calculate how the weather is going to develop in specific regions. One of the regional atmosphere model systems used by meteorological offices in several European countries is called Harmonie, and a simplified model used for teaching and education is called Harmonie MUSC.

The last four letters are an abbreviation for the French phrase Modèle Unifie Simple Colonne, which means that the calculations in the model are done in a single vertical “column”. The software was originally developed by the French Meteorological Institute, but is now being used in many European countries.

“The Harmonie MUSC software was excellent in itself, but experience taught us that it could be fairly laborious and time-consuming to install it on traditional desktop computers. The idea of using cloud services as a platform for column model software grew out of these challenges,” Niko Sokka explains.

Very satisfied

The Finnish Meteorological Institute and the CSC – IT Center for Science in Finland, where Glenna’s project manager Dan Still works, has a long history of collaborative efforts. So it was a natural step for the institutions to work together to further develop Harmonie MUSC into a cloud-based service. In the autumn of 2016, MUSC was successfully installed on CSC’s cPouta cloud service platform, in cooperation with the Norwegian and Swedish meteorological institutes: CloudMUSC was born.

“We are very satisfied with the cooperation and the results. This was a proof-of-concept involving half a dozen test users, and we have seen that it can be relatively easy to turn computer-based weather forecasting systems into cloud-based systems. In that sense, it was a very valuable information and learning exercise,” Sokka explains.

He adds that cloud-based systems are going to be increasingly important for European meteorological institutes. “There are several different approaches already ongoing in different countries, partly because we are moving towards Open Data systems. Weather forecasting research has been an international enterprise for many years, with a lot of data sharing, and cloud computing is obviously an important part of our future. The cooperation with Glenna contributed to opening the doors towards expanded use of cloud-based systems,” says Sokka.