LUMI-O Access and Authentication¶
Gaining access¶
To be able to use LUMI-O, you need to be a member of a LUMI project.
Create LUMI-O credentials¶
To connect to LUMI-O, one needs to create access credentials. These access credentials are valid for a pre-defined period of time. Note that if you are using the LUMI web interface, you can also create the access credentials directly in the web interface as described here.
Go to LUMI-O credentials management service page. And click on "-> Go to login".
Choose the correct authentication provider which for most LUMI users is "MyAccessID" (users with a Finnish allocation can also use "CSC" or "HAKA"), and follow the authentication procedure.
This page displays all the projects that are associated with your account. It shows the project number, project description, set storage quota, in GB, for a specific project, and how much of said quota is used up. Additionally, this page allows creating the necessary authentication keys, which are required for accessing the object storage.
Warning
The Usage column does not display correct data as the feature is still being implemented
To create an authentication access key pair. Open up the side menu from the rightward pointing arrow. In this example, we are opening the side menu for project 462000008, for which we will create the authentication key pair for.
The side menu allows for the creation of access keys. To generate an access key "Duration (hours)" and "Key description" fields must be filled out.
Tip
A good practice is to set the authentication key pair duration for the job walltime. This lets the job move the necessary data from LUMI-O to the scratch filesystem, perform the necessary calculations, and after finishing move the data back to LUMI-O.
When filling out the duration, keep the previous advice in mind.
Tip
The key description should be something relevant to the job it is meant for, that way it is easier to manage the created keys, should there be more than a few at the same time.
After the necessary fields are created, click on "Generate key". The generated key will appear in the side menu under "Available keys". The previously mentioned key description field is visible there, which makes it easy to distinguish between several keys.
Click on the newly generated Access key. This opens up the key's content. It provides the necessary "Access key" and "Secret key". You can also see the key description, which project said key is related to, owner of the key and finally Creation and Expiry dates.
From this side menu, it is also possible to extend the key.
Info
When you initially create the key, the initial allowed maximum lifetime is 168h = 7 days. This can then be extended by one hour for every hour from key creation, i.e., you are always able to extend the duration of the key to now + 7 days.
It's also possible to download a configuration template for different object storage clients like shell, boto3, rclone, s3cmd and aws. After selecting the desired object storage client and clicking "Generate" opens the output in a new browser tab.
Finally, by scrolling down in the menu this side menu allows you to delete the key. After deletion of a key, a new one needs to be created to resume utilizing LUMI-O for a certain project. Keys are non-recoverable, but a new one can be created in its place.
Connecting¶
You can access LUMI-O from any machine or server that is connected to internet. This can be a laptop, supercomputer, virtual machine in cloud or even your phone.
Using the LUMI web interface¶
LUMI web interface can be used to connect to LUMI-O.
On LUMI via terminal¶
With terminal, once logged in to LUMI, start by loading the lumio module which provides configuration and data transfer tools:
To configure a connection to LUMI-O, run the command:
This command asks you to connect with your browser to the LUMI-O credentials
management service, create credentials there and the copy
the project number and keys for the setup tool. The setup process will create configuration files for s3cmd
and rclone.
Read the step-by-step description for Creating LUMI-O credentials above.
Using the LUMI-O credentials management service, you can
also generate configuration for different object storage clients like shell,
boto3, rclone, s3cmd and aws. This is useful for using LUMI-O from somewhere
else than LUMI, where the lumio-conf command is not available (The tool can be
downloaded from the LUMI-O repository, but we only officially
support the tool on LUMI)
Other ways to access LUMI-O¶
Access without connecting to LUMI¶
In auth.lumidata.eu one can create configuration files associated to valid keys. These configuration files specify the details that are needed to access LUMI-O, and connecting to LUMI is actually not needed at all.
To connect to LUMI-O directly e.g. from your own laptop, create first a valid key in auth.lumidata.eu for a LUMI project. Then click the available "Access key", select a Configuration template format (rclone,s3cmd,..), and click "Generate". This opens a new tab with a template that you can use for a config file. See an example for rclone in Available tools.
Access via shell script¶
(explanation about LUMI-O access via shell script)