1. EndUser role

When running into some strange portal behavior, first try to logout/login from the portal before reporting an issue.

An EndUser can:

  • purchase an application from the MarketPlace,

  • manage datasets and Organizations in the Data Catalogue,

  • run application in Experiments

Some apps are freely available for all portal users. To use these apps, you can skip the purchase step. Right now all HiFiMagnet apps are freely available.

Before going to the Experiments it is mandatory to connect at least to Marketplace and to the Data Catalogue.

To use and create private datasets for storing inputs data and results, you have to upload your Data Catalogue key into the Experiments settings as described here.

1.1. Purchase an application

To purchase an application from the Marketplace you can either be DevUser or a simple EndUser. The process is the following:

  • connect to the Marketplace, in Home (top left Marketplace page) you will find all available apps (All categories).

  • click on Add to Cart to purchase the app you want.

  • click on Shopping Cart and Checkout to go on

    • fill the required forms for shipping address

    • provide a shipping address if not already defined

    • provide a billing address if not already defined

    • finalize your order by clicking on Checkout

To use the app, you will have to wait that the owner/seller of the app validate you order. To check the status of your order:

  • go to My Inventory

  • go to Product Orders

You shall see your products and their status in each product form.

To finalize your purchase DO NOT FORGET to connect to :

Then notify the owner/seller that you request to be part of HiFiMagnet user group. This will allow you to access HiFiMagnet singularity images and datasets.

To access HiFiMagnet docs you also need to have a user/password.

1.2. Data Catalogue

Any user of the platform can create and manage Organization in the Data Catalogue to deal with private datasets.

1.2.1. Create and Manage Organization

  • connect to Data Catalogue

  • click on Organizations

  • click on Add Organization

You, then, need to fill the form and finalize the creation by clicking on the create organization button.

You have then to manage user that are allowed to be part of your organization. They may have the following roles:

  • Admin : can view/add dataset, add user and manage role

  • Editor : can add dataset

  • Member : can view dataset

See the video bellow for a demo.

Users have to connect at least once to Data Catalogue before you can grant them a role in your organization.

1.2.2. Upload Data

To use private Organization and/or Dataset you would need to enter Data Catalogue key in the Settings menu. For instructions see here.

To add a dataset to an existing organization, you may need to ask the Admin of the organization to be granted the role of Editor to do so. In our case it is the owner/seller of HiFiMagnet app. to set <hpc_feelpp> to a specific directory for each instance.

1.3. Running an HiFiMagnet application in Experiments

*Experiments* *Instances* main menu for *EndUSer*

  • Go to Experiments

  • Select Add a new instance

    • define an ID

    • select a registered app in the sliding menu

At this point you should have a page with all the options that need to be filled.

ex: HiFiMagnet app `MagTherm`

The fields to fill may be grouped into categories:

  • specs for running simulations

  • HPC settings

  • specs retrieving singularity images

  • specs for singularity image

The EndUser shall only need to fill the fields related to the 1st and 2nd category. The fields that are to be filled by EndUser are marked in bold.

The others fields are designed for more advanced users.

Table 1. List of keys to be defined for being notified by email (only available in HiFiMagnet)
Key Description Default Notes

email_user

email address
Table 2. List of keys specs for running simulations
Key Description Default Notes

Dataset resource: model

dataset mode

None

comes from the Data Catalog

Output dataset: outputs_at

cfgfile

configuration file

/usr/share/doc/hifimagnet/ThermoElectricModel/quarter-turn3D.cfg

cfg files need to be in the container, or in a mounted directory or in the dataset
Table 3. List of keys to be defined for HPC settings
Key Description Default Notes

HPC: primary

HPC settings

"cesga"

hpc_modules

modules to load depending on the HPC machine

[gcc/6.3.0, openmpi/2.0.2, singularity/2.4.2]

depends on HPC and singularity image

hpc_basedir

volume where instance is created

${LUSTRE}

depends on HPC and singularity image

hpc_feelpp

volume where results will be stored

${LUSTRE}/feel

depends on HPC and app

hpc_volumes

volumes to be mounted on the HPC machine

[/scratch,/mnt,${LUSTRE}/feel:/feel]

depends on HPC and singularity image

hpc_partition

select partition queue

'thin-shared'

shall depends on batch system and HPC

hpc_reservation

select reservation queue

``

optional

parallel_tasks

number of tasks/processes to run in parallel

2

depends on batch system and selected partition

max_time

maximum allowed time

'00:30:00'

hours, minutes and seconds
Table 4. List of keys specs for retrieving singularity images
Key Description Default Notes

sregistry_storage

path to container directory

${LUSTRE}/singularity_images

HPC settings for local Lncmi ressources are specificied here.

Once filled, click on Create New Instance.

While creating a new app instance, any error will result in a pop-up message.
ex: Running HiFiMagnet `MagTherm`

The next step is to launch the app:

  • select the instance ID, you defined in the previous step,

  • click on Run, wait until the button in blue turns green.

Then you have access to:

  • the execution logs: monitor the workflow (deploying the instance, running the boostrap/revert script and executing the tasks),

  • the applications logs: monitor specific logs for the instance ran.

The result are stored in <hpc_feelpp>. Make sure that <hpc_volumes> bindmount list option contains this field: eg <hpc_feelpp>:/feel on cesga where <hpc_feelpp>=$LUSTRE/feel.

As a best practice, we highly recommend the user to set <hpc_feelpp> to a specific directory for each instance.

It means that the actual simulation results are stored in:

  • <hpc_feelpp>/hifimagnet/ThermoElectricModel/<geofile>/thermoelectric-linear_V1T1_N1/np_<parallel_tasks> for MagThel app.

<geofile> correspond to the input with the same name in the <cfgfile>.

When running the instance ID:

  • a directory <hpc_basedir>/<workdir_prefix>_<date>_<id> will be created (`<id> is a random number);

  • the dataset will eventually be downloaded and extracted into this directory;

  • the script <job_prefix><sid>.script will be created and submitted to the queueing system of the HPC ressource.

Upon completion two files will be created in the directory:

  • <job_prefix><sid>.out: contains the output of the script,

  • <job_prefix><sid>.err: contains some log and errors if any.

A red button will indicates an error. In this event, check the output of the script (aka slurm-<id>.out, the .err and .out files). To do so you need to have <workdir_keep> set to true.

Finaly to destroy your instance ID , just click on:

  • Destroy Instance!

  • or Force Destroy

This will remove your instance ID from the instance list.

See this video for a live demo.

1.4. Running an HiFiMagnet application in Experiments using datasets from DataCatalogue

Basically the procedure is the same as above. The only difference here is that you need:

  • to select a dataset in the DataSet sliding menu

  • to select a ressource within the DataSet

  • to enter the name of the <cfgfile> you will use

You should at the end have something like in the following image.

Selecting a dataset

The dataset, at least those from HiFiMagnet organization in the Data Catalogue are gzipped archive that holds all the files needed to run:

  • a mesh or cad file

  • a <cfgfile>

  • a list of json files for physical properties and/or boundary conditions

  • eventually some more files when using some MagnetTools features (eg …​)

The format of the files in a dataset are respectively described in:

  • for HiFiMagnet simulation

  • for mesh and cad format

  • for MagnetTools

1.5. Running an Remote Desktop resource in Visualization

  • Go to Visualization

  • Click Desktop Available

    • Select your Infrastructure in the sliding menu

If no Remote desktop is defined, you will receive a warning message. Create a Setting, then go back to this step.

  • Click on Create

  • Click either on Desktop or View Only

It should start a web page within your browser. In this page you should be able to:

  • start a 'terminal' and check the output of your simulation

The outputs of the job (ie slurm jobs) are stored in the <hpc_basedir>/<hpc_workdir_prefix>_<date>_<id>. The actual results for the simulation would be stored in <hpc_feelpp> directory following Feel++ usage.

  • start a post-processing GUI:

  • to quit just click on Menu/logout (top left)

For more details on using ParaView or EnSight please check this section. See the video bellow for a live demo.

You can also perform some pre-processing GUI with Salome HiFiMagnet plugin. See here for details.

2. To go further

Available apps in MSO4SC Portal are:

To get more details about the app, click on it. This will display more information about the app and its settings.