Page Comparison

...

Note

title	New method available

There is now a free cloud process that can be used to build from a recipe that does not require root. This process still uses all of the information from this page but adds some additional steps which we have documented at Using the Sylabs Builder tool

What is a container

Containers are similar to virtual machines. They allow a user to run software that is otherwise incompatible with the native operating system. For example, some scientific software is written to only be compatible with Ubuntu, but our systems only run CentOS. Running the software inside a container provides the software an Ubuntu environment without re-imaging a cluster node.

...

Containers further have a huge advantage in that they are portable. Once you have a singularity image created, it will run on any other system that has singularity installed. A Mac, CentOS, Ubuntu box, even other clusters will run your Singularity container with no modification. You could take your singularity image and it would run perfectly on any other HPC cluster with Singularity installed.

What is Singularity and why not Docker?

Singularity is a container manager designed with HPC environments such as ASU's in mind.

...

Singularity does not run the container network transparently like docker. Network ports must be specifically forwarded on the command line when initiating an image
The ASU RC use policy does not permit running servers on nodes, therefore many ports are blocked on the nodes and network.
Small exceptions are provided for important functions such as databases to support the analysis software. Generally, these services cannot bind to all IP's of a compute node, and are limited to only accessing the database server locally (cannot be shared between nodes)
The above limitations means images that attempt to run servers or bind ports of the native system may not work

Basic usage

In order to pull and use a container as-is

...

Log into agave ( ssh asurite@agave.asu.edu )
Start an interactive session or add commands as needed to an SBATCH script
load one of the singularity modules ( module load singularity/3.3.0-1 )
Pull the image e.g. from docker for the biobakery lefse image https://hub.docker.com/r/biobakery/lefse ( singularity pull docker://biobakery/lefse )
run the image ( singularity run lefse_latest.sif )
run your commands as you normally would inside the container

Importing a path

We don't want to have to copy all of our files into a singularity image. Instead, it's often easier to just give Singularity access to a path within the container.

...

Code Block

language	bash

[user@cg1-2:~]$ module load singularity/3.3.0-1
[user@cg1-2:~]$ singularity run lefse.sif
[user@cg1-2:~]$ cd /scratch/user
bash: cd: /scratch/user: No such file or directory
[user@cg1-2:~]$ exit
[user@cg1-2:~]$ singularity run -B /scratch/user:/scratch/user lefse.sif
[user@cg1-2:~]$ cd /scratch/user
[user@cg1-2:/scratch/user]$ ls -la
total 10774371
drwxrwxr-x 2 user user          3 Dec  2 12:01 .
drwxr-xr-x 3 user user         60 Feb 10 09:40 ..
-rw-r--r-- 1 user user 8157644800 Dec  2 12:04 E6B-113X.tar
-rwxr-xr-x 1 user user  922681375 Sep 25 14:35 lefse.simg
-rw-rw-r-- 1 user user 1952628224 Nov  4 13:01 nagiosxi-5.6.7-64.ova
[user@cg1-2:/scratch/user]$ exit

Connecting to a GPU

Singularity will work with GPU's, however, when running the singularity image you will need to include an extra switch "--nv"

e.g.
singularity run --nv /packages/7x/vmd/1.9.3/vmd.sif

Advanced usage

In the event you need to work with a container that requires you to customize an image, you will need to perform additional steps.

...

To install on Linux systems follow the directions at: https://sylabs.io/guides/3.3/user-guide/quick_start.html#quick-installation-steps
Installation for Mac with either a DMG or Homebrew: https://sylabs.io/guides/3.3/user-guide/installation.html#install-on-windows-or-mac
Windows users will need to install several applications. See our Windows Singularity documentation for details

Building from a recipe

Info

The below build is provided only as a practical example of a successful Singularity build used on Agave.

You will need to modify the build script to match the software and container OS you will use

...

Info
This document only covers specifics for ASU HPC and does not constitute complete information on using singularity. For complete information visit the Singularity documentation at https://sylabs.io/docs/

Using the Sylabs builder tool

A separate write-up for the cloud builder tool can be found here.

Converting Dockerfiles/Images

Please note that depending on how the Docker image was created it may not run perfectly without additional singularity command line options. These options will be unique to each docker image. Consult the documentation for the Docker Image and cross reference with Singularity documentation to determine which options to use.

...