HiFiMagnet user manual

Welcome to HiFiMagnet manual documentation!

The HiFiMagnet manual documentation introduces the process to build and run an application. New user shall read the HiFiMagnet introduction to have a global view.
You wish to contribute? Please consult the contribute section to learn how to proceed!

Introduction to HiFiMagnet

Discuss and Contribute
Use Issue 858 to drive development of this section. Your contributions make a difference. No contribution is too small.

What is HiFiMagnet?

HiFiMagnet is a software chain that allows to design and model High Field Magnets. It consists in a set of tools including:

When using containers, note that the tools are distributed into 3 differents containers:

  • one for pre-processing (see here),

  • one for running simulations (see respectively here for Axi and here for 3D),

  • one for post-processing (see here).

EnSight (recommended)

is a commercial multi-platform data analysis and visualization application.

ParaView

is an open-source, multi-platform data analysis and visualization application.

Visit

is a distributed, parallel visualization and graphical analysis tool for data defined on two- and three-dimensional (2D and 3D) meshes

Quick Starts

1. Getting HiFiMagnet

Using HiFiMagnet inside container, either Docker or Singularity based, is the recommended and fastest way. If your system support singularity containers we recommend to use them instead of Docker ones.

1.1. from sregistry

To get the HiFiMagnet singularity image you need first to get you token on a valid sregistry:

  • connect to the Sregistry service (eg cesga sregistry use the FiWare authentification}

  • in the top right scrolling menu with your user id, select Token

  • copy the line to a .sregistry in your home directory

The HiFiMagnet singularity images are stored in a private collection. To download the images you have to be granted access to the HiFiMagnet team. Once this is done, from a terminal:

export SREGISTRY_CLIENT=registry
export SREGISTRY_CLIENT_SECRETS=~/.sregistry-cesga
[export SREGISTRY_STORAGE=...]

sregistry pull --name hifimagnet-stretch.simg hifimagnet/hifimagnet:stretch

1.2. from dockerhub

You need to be member of Feel++ team on dockerhub to be able to download the image. To get the image:

docker login
docker pull feelpp/hifimagnet

Check the existing docker images on feelpp/hifimagnet dockerhub

1.3. Install from Lncmi Debian repository

On Debian/Ubuntu system you can install HiFiMagnet from the local Lncmi packages repository. To do so:

  • you need to have access to the Lncmi repository

  • add a lncmi.lst into your /etc/apt/sources.list.d directory:

lncmi.lst for Debian Testing distribution
deb http://euler/~trophime/debian/ testing main
deb-src http://euler/~trophime/debian/ testing main

Then run:

sudo apt update
sudo apt install hifimagnet

Supported Debian/Ubuntu distributions:

  • Stretch

  • Xenial

On Windows 10 Pro you can also install HiFiMagnet without much effort. Just enable WSL feature and download Debian Stretch or Ubuntu Xenial (aka 16.04 LST) from Microsoft App Store. Then apply the same procedure as above to install the package.

2. Running HiFiMagnet

2.1. from Singularity container

Assuming we use hifimagnet.simg singularity image:

singularity run -it -v $HOME/feel:/feel feelpp/hifimagnet feelpp_hfm_coupledmodel_3DP1N1 --config  ...

The coupled_3D_P1_N1_singular_cvg.cfg configuration file holds all the information to specify what calculations are performed. The configuration files are fully described in this section

2.2. from Docker container

Start the Docker container feelpp/hifimagnet as follows

docker run -it -v $HOME/feel:/feel feelpp/hifimagnet feelpp_hfm_coupledmodel_3DP1N1 --config  ...

2.3. from command line

To perform a 3D fully coupled multi-physics simulation:

feelpp_hfm_coupledmodel_3DP1N1 --config coupled_3D_P1_N1_singular_cvg.cfg

Using HiFiMagnet

As stated in introduction HiFiMagnet per se provides tools to simulation High Field Magnets. More precisely we can model:

  • thermo-electric behavior,

  • magnetic field produced:

    • everywhere

    • only in a zone of interest from a user perspective

  • mechanical behavior accounting for:

    • an axisymetrical estimation of the magnetic field

    • the magnetic field produced

    • thermal dilation

  • fully coupled behavior

The kind of calculation performed is specified in a configuration cfg file. The data settings consist in writting json files for each physical model involved, providing connection between CAD/Mesh entities to respectively Material and Boundary conditions.

1. Data structure

HiFiMagnet simulations parameters are defined througth configuration file .cfg and json files defining the model per physic and eventually some other json files for the physical properties of the materials.

The data structure follows Feel++ style and convention. In this section we will give details about these different files.

1.1. Config Files

dim=3
geofile=quarter-torus3D.geo (1)
geofile-path=$cfgdir
gmsh.scale=0.001 (2)
gmsh.hsize=5

conductor_volume=coil (3)
#insulator_volume=glue

compute_magnetism=true (4)
compute_bg_magfield=false
compute_bg_magfield_bmap=false
compute_elasticity=false

[convergence]
max_iter=1

[functions]
j={58.e+3*(0.5/(2*Pi))*y/(x^2+y^2),-58.e+3*(0.5/(2*Pi))*x/(x^2+y^2),0}:x:y:z
v=0.5*atan2(y,x)/(2*Pi)*(atan2(y,x)>0)+(0.5*(atan2(y,x)+2*Pi)/(2*Pi))*(atan2(y,x)<0):x:y:z
t=362.156146169164-58.e+3/(2*0.38)*(0.5/(2*Pi))^2*log(sqrt(x*x+y*y)/39.4354779237947 )^2:x:y:z

[thermoelectric] (5)
model_json=$cfgdir/quarter-torus3D-therm.json
weakdir=false

[electro]
pc-type=boomeramg #gamg
#ksp-monitor=true
ksp-rtol=1e-7
ksp-atol=1e-5
ksp-maxit=2000
ksp-use-initial-guess-nonzero=1

[thermal]
pc-type=boomeramg #gamg
#ksp-monitor=true
ksp-rtol=1e-8
ksp-atol=1e-6
ksp-use-initial-guess-nonzero=1

[elasticity] (6)
filename=$cfgdir/quarter-torus3D-elasticity.json
on.type=elimination_symmetric
lorentz_forces=true
thermal_dilatation=false
# # precondtioner config
pc-type=boomeramg #gamg #lu,gasm,ml
ksp-monitor=true
# ksp-converged-reason=1

[magnetic_field-bmap] (7)
geo-data=torus3D.d
geo-path=$cfgdir
helix-intensity=0
bitter-intensity=11767.7
supra-intensity=0

[magnetostatic] (8)
model_json=$cfgdir/quarter-torus3D-mag.json
weakdir=false
eps-coeff=0
# print-info=1

[ms]
pc-type=ams
#
# pc-hypre-ams options
# see : http://www.mcs.anl.gov/petsc/petsc-dev/src/ksp/pc/impls/hypre/hypre.c.html
#
# when ams is used as a preconditionner
# use ams-cycle-type : 2,4,6,8,12,14
#
# relax-type:
# 1 weighted Jacobi relaxation,
# 2 a hybrid Gauss-Seidel / Jacobi relaxation scheme,
# 3 a symmetric hybrid Gauss-Seidel / Jacobi relaxation scheme,
# 4 l1-Gauss-Seidel or Jacobi,
# 5 Chebyshev smoothers,
# 6 hybrid block and Schwarz smoothers [36],
# 7 ILU and approximate inverse smoothers.
#
# when ksp-type=cg
# use a symmetric smoother such as
# weighted Jacobi or hybrid symmetric Gauss-Seidel
pc-hypre-ams-max-iter=15
pc-hypre-ams-print-level=2
pc-hypre-ams-cycle-type=13
pc-hypre-ams-tol=1.e-6
pc-hypre-ams-relax-type=4
pc-hypre-ams-relax-times=4
pc-hypre-ams-relax-weight=0.8
pc-hypre-ams-omega=0.8
#
pc-view=1
ksp-type=preonly #cg
ksp-monitor=1
ksp-atol=2e-3
ksp-rtol=1e-7
ksp-verbose=1
ksp-view=0
show-preconditioner-options=1
#ksp-use-initial-guess-nonzero=1

# in json:    "Model": "MagnetoStatic-singular"
[ts]
steady=true
1 name of the geometry or mesh
2 gmsh options: scale (optional, the default is meter), mesh size
3 list of conductor and insulator volumes
4 flag to control calculations
5 thermo-electric model
6 elasticity model
7 magnetostatic model
8 specific to HiFiMagnet

The first lines define the geometry or mesh to consider:

  • name of the geometry or mesh

  • gmsh options: scale (optional, the default is meter), mesh size

  • list of conductor and insulator volumes

The next section give setup the calculations to be ran:

This is followed by section dedicated to specific physics. In this example we have:

  • thermo-electric model

  • elasticity model

  • magnetostatic model

Each of this physic section contains:

  • name of the json file holding the complete description of the model (see bellow),

  • setup of the solver: eg [electro] and [thermal] for thermo-electric model

Solver are specific to each physical model. However in general, the default solver is a direct solver. For large problems, we would prefer iterative solver for efficiency. A typical iterative solver is defined as follow:

  • pc-type: name of the preconditionner,

  • ksp-rtol: relative tolerance to stop iterative solver,

  • ksp-atol: absolute tolerance to stop iterative solver,

  • ksp-maxit: maximum iterations allowed to reach convergence,

  • ksp-use-initial-guess-nonzero: enable to start with a non null initial guess solution.

Section magnetic_field-bmap is specific to HiFiMagnet. It enables to define data needed to compute the "ideal" magnetic field provided by a magnet using xref:magnettools:Bmap.adoc#bmap:

  • the .d cfg file describing the Axisymetrical model of the magnet

  • the values for the input currents in each subsets of the magnet

A magnet is composed of PolyHelices insert and eventually some external magnets either Bitter magnets and/or Supraconductor Magnets.

1.2. Physic Model files

The Physic Model files are json files that contain informations about:

  • Materials

  • Boundary Conditions

  • Post-Processing quantities

1.2.1. ThermoElctric Model file

{
    "Name": "CoupledCart", (1)
    "ShortName":"MSC",
    "Models": (2)
    {
        "equations": "thermoelectric-linear"
    },
    "Materials": (3)
    {
        "coil":
        {
            "name":"copper",
            "filename":"$cfgdir/Cu.json"
        }
    },
    "BoundaryConditions": (3)
    {
        "potential": (4)
        {
            "Dirichlet":
            {
                "V0":
                {
                    "expr1":"0.5/4.",
		    "expr2":"coil"
                },
                "V1":
                {
                    "expr1":"0",
		    "expr2":"coil"
                }
            }
        },
        "temperature": (5)
        {
            "Robin":
            {
                "Rext":
                {
                    "expr1":"80000",
                    "expr2":"293"
                },
                "Rint":
                {
                    "expr1":"80000",
                    "expr2":"293"
                }
            }
        }    },
    "PostProcess": (6)
    {
        "Exports":
        {
            "fields":["temperature","potential","joules","current"]
        }
    }
}

1.2.2. Magnetostatics Model file

{
    "Name": "CoupledCart", (1)
    "ShortName":"MSC",
    "Models": (2)
    {
        "equations": "magnetostatic-regularized"
    },
    "Materials": (3)
    {
        "coil":
        {
            "name":"copper",
            "mu_mag": "1"
        },
        "air":
        {
            "name":"air",
            "mu_mag": "1"
        }
    },
    "BoundaryConditions": (4)
    {
        "magnetic_potential":
        {
            "Dirichlet":
            {
                "Border":
                {
                    "expr":"{0,0,0}:x:y:z"
                },
                "OYOZ":
                {
                    "expr":"0:x:y:z"
                },
                "V1":
                {
                    "expr":"0:x:y:z"
                },
                "OXOZ":
                {
                    "expr":"0:x:y:z"
                },
                "V0":
                {
                    "expr":"0:x:y:z"
                }
            }
        }
    },
    "PostProcess":, (5)
    {
        "Exports":
        {
            "fields":["magneticPotential","magneticField","currentDensity"]
        }
    }
}

1.2.3. Elasticity Model file

{
    "Name": "CoupledCart", (1)
    "ShortName":"MSC",
    "Models": (2)
    {
        "equations": "Elasticity"
    },
    "Materials": (3)
    {
        "coil":
        {
            "name":"copper",
            "E":"2.1e6",
            "nu":"0.33",
            "alphaT":"18e-6",
            "rho":"10.e-6"
        }
    },
    "BoundaryConditions": (4)
    {
        "displacement_z":
        {
            "Dirichlet":
            {
                "HP":
                {
                    "expr":"0"
                },
                "BP":
                {
                    "expr":"0"
                }
            }
        },
        "displacement_y":
        {
            "Dirichlet":
            {
                "V0":
                {
                    "expr":"0"
                }
            }
        },
        "displacement_x":
        {
            "Dirichlet":
            {
                "V1":
                {
                    "expr":"0"
                }
            }
        }
    },
    "PostProcess": (5)
    {
        "Exports":
        {
            "fields":["displacement","Von-Mises","tresca","principal-stresses"]
        }
    }
}

1.3. Materials file

Physical properties of the material are defined in a json file for each material. The properties are given in SI units.

{
    "name":"Cu", (1)
    "sigma0":"50.e+6",
    "k0":"330",
    "T0":"293",
    "alpha":"3.4e-3", (2)
    "sigma":"sigma0/(1+alpha*(T-T0)):sigma0:alpha:T:T0", (3)
    "k":"k0*T/((1+alpha*(T-T0))*T0):k0:T:alpha:T0", (4)
    "E":"120.e+9", (5)
    "nu":"0.35", (6)
    "alphaT":"18e-6", (7)
    "rho":"8.96e+3", (8)
    "mu":"1" (9)
}

In the table bellow, you will find the correspondance between an entrie in the json file defining properties for material <1> :

Notation

Quantity

Unit

Note

<2> \(\alpha\)

thermal resistivity coefficient

-

<3> \(\sigma\)

electrical conductivity

\(S.m^{-1}\)

an expression of \(T\)

(references values,indiced with a \(_0\), at \(T_0\))

<4> \(k\)

thermal conductivity

\(W.m^{-1} .K^{-1}\)

same

<5> \(Y\)

Young modulus

\(Pa\)

<6> \(\nu\)

Poisson ratio

-

<7> \(\alpha_T\)

thermal expansion coefficient

\(K^{-1}\)

<8> \(\rho\)

material’s density

\(kg.m^{-3}\)

<9> \(\mu\)

magnetic permeability

\(V.s.A^{-1}.m^{-1}\)

2. Workflow

2.1. Using MagnetTools to define Design

  • create optimized axi geometry

  • run opt2yml to create the cfg files for HiFiMagnet Salome Plugin

2.2. Using HiFiMagnet Salome Plugin to create CAD and generate meshes

  • fill in the missing data into Salome yaml cfg files

  • run the CAD generation

  • eventually add Air

  • mesh

  • improve the mesh

2.3. Prepare the mesh for HPC

The mesh created with HiFiMagnet Salome is in med format. Depending on the version of Feel++ used this format may not be supported. So before running any simulation you have to convert the mesh to gmsh V2 format:

gmsh -bin -3 mesh.med -o mesh.msh

Then you have eventually to partition the mesh to use it on HPC:

feelpp_mesh_partitioner [--gmsh.scale=0.001] --ifile mesh.msh -ofile mesh-np4 --part 4 --nochdir

where part denotes the number of processors (4 in our example) you want to use. The option --gmsh.scale=0.001 enables to rescale the mesh file.

This will create new files mesh-np4.json and mesh-np4.h5 ready for running.

HiFiMagnet Salome CAD geometries are defined in mm. The mesh file generated is also in mm.

As a consequence, do not forget to rescale the mesh before running simulation. Use --gmsh.scale=0.001 to rescale the mesh.

2.4. Define Cfg, Model and Materials files

2.5. Running 3D simulations

2.6. Postprocessing results

3. Examples