Versions Compared

Version Old Version 6 New Version Current
Changes made by

tianche5

tianche5

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

...

...

...

Panel
panelIconId1f40d
panelIcon:snake:
panelIconText🐍
bgColor#E3FCEF

This example is about installing and using one Python package in a Mamba environment.

Table of Contents
minLevel1
maxLevel6
outlinefalse
stylenone
typelist
printabletrue

...

Install and use a python package called multiqc on Sol.

...

Multiqc

Go to anaconda.org and search for multiqc in the search bar:

...

Click on one of the search results, usually the first one:

...

The “conda install” on the above page gives the installation commands. :

...

The conda part in the red circle needs to be changed to mamba, and the bioconda part in the blue box is the channel name of this multiqc package. Channel is similar to the name of an online folder, that mamba can find and download the correct package. This is important for the next step.

The command given here CANNOT be directly used on ASU supercomputers, the sections below will show you how to re-write an install command.

Step

...

2 - Install

...

  1. Connect to the VPN

  2. Open a command line interface on

...

  1. Sol by navigating to sol.asu.edu on your browser and
    selecting the "Sol Shell Access" option from the "System" menu option. Or by SSHing into
    Sol using the command ssh <asurite>@sol.asu.edu

...

  1. interactive -p htc -c 4 -t 30

...

  1. module load mamba/latest

5. mamba create -n myENV -c bioconda multiqc

  1. If the package is for running codes on GPUs, a compatible cuda module is also needed.

  2. Now we are ready to create env and install packages. It can be done in separated commands, but the fastest way is to do it in a single command, here is how:

    1. Create an environment:

      Code Block
      mamba create -n myENV -c conda-forge python=3
source activate myENV

    2. Install Multiqc using the command found in step 1

      Code Block
      mamba install -c bioconda multiqc

    3. Combine commands from a, b, c into the fastest single command:

      Code Block
      mamba create -n myENV -c conda-forge -c bioconda python=3 multiqc

  3. And answer Y to the promoted question, if everything looks fine

...

  1. Wait till the installation finish

The -c flag in the above command means “channel". The channel name must be correct to install the correct package. In our case, the channel of multiqc is bioconda, as found in Step Two.

...

More information can be found here: Managing Python Modules Through the Mamba Environment Manager

Step 3 - Use

Once the myENV environment is ready, multiqc can be used directly or within a python session/script.

  1. Connect to the VPN

  2. Open a terminal app or a command line interface on a personal laptop or pc, or in a Sol desktop session. And then type in the commands:

...

  1. Sol by navigating to sol.asu.edu on your browser and
    selecting the "Sol Shell Access" option from the "System" menu option. Or by SSHing into
    Sol using the command ssh <asurite>@sol.asu.edu

  2. interactive -p htc -c 4 -t 30

  3. module load mamba/latest

  4. source activate myENV

  5. python

  6. import multiqc

Step

...

4 - What about pip?

As explained in Python Package Installation Method Comparison , if a package is not found on anaconda.org, but found on pypi.org , pip can be used inside a mamba env. This is the only correct way to use pip on the ASU supercomputers , the only correct way to use pip on the ASU supercomputers at the moment, is to use it inside an activated mamba env.

There are some packages can only be found on pypi.org but not anaconda.org, then the only option to install them is via pip, inside an activated mamba env. Notably that the current official installation guide for both tensorflow and pytorch prefers pip, but the conda-forge channel still supports them, and our public tensorflow/pytorch environments were built via mamba.

Panel
panelIconId26d4
panelIcon:no_entry:
panelIconText⛔
bgColor#FF8F73

Almost all of the Python packages can be found on anaconda.org, and mamba is always preferred.

...

The the Here we take a python package called q2-greengenes2 as an example. The commands to have this package it installed via pip to a mamba env are:

  1. Open a terminal app or Connect to the VPN

  2. Open a command line interface on a personal laptop or pc, or in a Sol desktop session. And then type in the commands:

...

  1. Sol by navigating to sol.asu.edu on your browser and
    selecting the "Sol Shell Access" option from the "System" menu option. Or by SSHing into
    Sol using the command ssh <asurite>@sol.asu.edu

  2. interactive -p htc -c 4 -t 30

  3. module load mamba/latest

  4. Two options here

    1. If you have an existing env: source activate myENV

    2. If you want to build a new env:

      Code Block
      mamba create -n myENV -c conda-forge python=3
source activate myENV

  1. pip install q2-greengenes2

Step 5 - Jupyter Notebook

After multiqc and q2-greengenes2 have been installed to myENV, we want to use this mamba env in the Jupyter Notebook session on the Sol web portal. So we need to make a Jupyter kernel from this mamba env. More details are covered in Preparing Python Environments for Jupyter and here are the example steps:

  1. Connect to the VPN

  2. Open a command line interface on Sol by navigating to sol.asu.edu on your browser and
    selecting the "Sol Shell Access" option from the "System" menu option. Or by SSHing into
    Sol using the command ssh <asurite>@sol.asu.edu

  3. interactive -p htc -c 4 -t 30

  4. module load mamba/latest

  5. mkjupy myENV "myENV_kernel"

  6. Find and use myENV_kernel

    1. Log in to the Sol web portal

    2. On the top bar: Interactive Apps > Jupyter > Fill out request form > Connect to Jupyter

    3. Inside the Jupyter Notebook: Open a Launcher page > Click on myENV_kernel icon