/
A Brief Example

A Brief Example

This tutorial is about installing and using two Python packages in a Mamba environment.

Step 1 - Search

Example packages used for this tutorial are: multiqc and tensorflow. Step 1 is about finding multiqc. Step 4 will cover tensorflow.

Go to anaconda.org and search for multiqc in the search bar:

Screenshot 2024-11-13 at 1.53.54 PM.png

Click on one of the search results, usually the first one:

Screenshot 2024-11-13 at 1.55.18 PM.png

Exam the details on the following page, especially the version of the package:

Screenshot 2024-11-13 at 1.52.12 PM.png

The “conda install” on the above page gives the installation commands:

Screenshot 2024-11-13 at 1.58.33 PM.png

The conda part in the red circle needs to be changed to mamba, and the bioconda part in the blue box is the channel name of this multiqc package. Channel is similar to the name of an online folder, that mamba can find and download the correct package. This is important for the next step.

The command given here CANNOT be directly used on ASU supercomputers, the sections below will show you how to produce a correct install command.

 

Step 2 - Install

  1. Connect to the VPN

  2. Open a command line interface on Sol by navigating to sol.asu.edu on your browser and
    selecting the "Sol Shell Access" option from the "System" menu option. Or by SSHing into
    Sol using the command ssh <asurite>@sol.asu.edu

  3. interactive -p htc -c 4 -t 30

  4. module load mamba/latest

  5. If the package is for running codes on GPUs, a compatible cuda module is also needed.

  6. Now we are ready to create env and install packages. It can be done in separated commands, but the fastest way is to do it in a single command, here is how:

    1. Slower way:

      1. Create an environment:

        mamba create -n myENV -c conda-forge python=3 source activate myENV

      2. Install Multiqc using the command found in step 1

        mamba install -c bioconda multiqc

    2. Faster way: Combine commands from above into the a single command:

      mamba create -n myENV -c conda-forge -c bioconda python=3 multiqc

  7. And answer Y to the promoted question, if everything looks fine

  8. Wait till the installation finish

The -c flag in the above command means “channel". The channel name must be correct to install the correct package. More information can be found here: Managing Python Modules Through the Mamba Environment Manager

 

Step 3 - Use / Test

Once the myENV environment is ready, multiqc can be used directly or within a python session/script. Below is for using it in the shell, which is also a good testing method for newly built env.

  1. Connect to the VPN

  2. Open a command line interface on Sol by navigating to sol.asu.edu on your browser and
    selecting the "Sol Shell Access" option from the "System" menu option. Or by SSHing into
    Sol using the command ssh <asurite>@sol.asu.edu

  3. interactive -p htc -c 4 -t 30

  4. module load mamba/latest

  5. source activate myENV

  6. python

  7. import multiqc

 

Step 4 - What about pip?

As explained in Python Package Installation Method Comparison , the only correct way to use pip on the ASU supercomputers at the moment, is to use it inside an activated mamba env.

There are some packages can only be found on pypi.org but not anaconda.org, then the only option to install them is via pip, inside an activated mamba env. Notably that the current official installation guide for both tensorflow and pytorch prefers pip.

Screenshot 2025-04-11 at 2.15.36 PM.png

Here we take tensorflow as an example. We will need to modify the instructions found on their official document to make it work on Sol:

  1. Connect to the VPN

  2. Open a command line interface on Sol by navigating to sol.asu.edu on your browser and
    selecting the "Sol Shell Access" option from the "System" menu option. Or by SSHing into
    Sol using the command ssh <asurite>@sol.asu.edu

  3. interactive -p htc -c 4 -t 30

  4. module load mamba/latest

  5. Since we are on a CPU compute node right now, we need CUDA drivers so the env we are making can used GPU nodes in the future. That means we need a CUDA module:
    module load cuda-12.6.1-gcc-12.1.0

  6. Now we need to go inside an env. Two options here:

    1. If you have an existing env: source activate myENV

    2. If you want to build a new env:

      mamba create -n myENV -c conda-forge python=3.12 source activate myENV

  7. pip install tensorflow[and-cuda]

 

Step 5 - Jupyter Notebook

After multiqc and tensorflow have been installed to myENV, we want to use this mamba env in the Jupyter Notebook session on the Sol web portal. So we need to make a Jupyter kernel from this mamba env. More details are covered in Preparing Python Environments for Jupyter and here are the example steps:

  1. Connect to the VPN

  2. Open a command line interface on Sol by navigating to sol.asu.edu on your browser and
    selecting the "Sol Shell Access" option from the "System" menu option. Or by SSHing into
    Sol using the command ssh <asurite>@sol.asu.edu

  3. interactive -p htc -c 4 -t 30

  4. module load mamba/latest

  5. Create a jupyter kernel, note that you don't need to activate any environment.

    mkjupy myENV "myENV_kernel"

  6. Find and use myENV_kernel

    1. Log in to the Sol web portal

    2. On the top bar: Interactive Apps > Jupyter > Fill out request form > Connect to Jupyter

    3. Inside the Jupyter Notebook: Open a Launcher page > Click on myENV_kernel icon. It usually shows up as the first cube, in front of the public kernels:

Screenshot 2025-03-04 at 9.14.08 AM.png

Once a jupyter kernel is made, it cannot be modified. So if you need to add more packages later, the correct method is:

  1. Open a shell/ terminal to access Sol or Phx

  2. Add the packages to the existing mamba env

  3. Recreate the jupyter kernel using the mkjupy command. You can use a new name if you want to keep the old kernel.

  4. Launch a new jupyter session on the Web Portal, and look for this new kernel.

 

Additional Help