...

Your first time using a supercomputer

...

like Sol can be intimidating, but it

...

does not have to be. This guide will get you started with the basics. If you run into problems or need additional help, we hold regular weekly office hours.

...

...

...

This document also assumes you already have requested and been granted an account. If not, please see Getting an Account

Please also familiarize yourself with our Acceptable Use Policy before getting started. Sections 4-8 contain important information on permitted and prohibited activities. If not all the terms make sense yet don’t worry, we’ll be covering it further down in this document.

If you run into problems or need additional help, we hold regular weekly office hours (Holiday & Summer hours may vary).

...

Quick Start

For users who have never used a supercomputer before, we recommend reading through the “Detailed Start” section of this document.

For those who wish to get started quickly, here is the general overview:

1. Connect to the ASU Cisco AnyConnect VPN

2. Login with your ASURITE & password

3. Choose a connection method (terminal / web portal)

4. Transfer needed files

5. Run an interactive session or create an SBATCH script

6. Access mamba for Python environments, other public software modules, and /wiki/spaces/RC/pages/1754857495

Important Terms

• HPC: Short for “High Performance Computing” it refers to a group (or a cluster) of interconnected computers designed for parallelism across many computers at once. Publicly these are often called “supercomputers”.

• Node: A single machine in a supercomputer. This will be either a physical machine or a virtual machine. 

• Login Node: A node intended as a launching point to compute nodes. Login nodes have minimal resources and should not be used for any application that consumes a lot of CPU or memory. Also This is also known as a head “head node”.

• Compute Node: Nodes intended for heavy computing. This is where all heavy processing should be done

• RC: Short for Research Computing, the team that manages the ASU HPC supercomputer.

• HPC: Short for “High Performance Computing” it refers to a group (cluster) of computers designed for parallelism across many computers at once. Publicly these are often called “supercomputers”

• Cluster: A group of interconnected computers that can work cooperatively or independently.

• Job: Work assigned to be done on a compute node. Any time a compute node is assigned a job is created.

• Memory (RAM): Short for “Random-Access Memory“. This is used for the amount of memory that each calculation or computation requires in order to execute and complete successfully. The term “memory“ is not used for disk space. This is another main component that defines a node.

• CPU: Short for “Central Processing Unit”, also called a core. This is one of the main components that defines a computing device, such as a node.

• GPU: Short for “Graphic Processing Unit”. This is a specialized piece of hardware that can enable and accelerate certain computational research.

• Scheduler: The application on our end that manages and assigns (allocates) compute resources for jobs. Slurm: The brand name of our scheduler like which manages and allocates resources.

Quick Start

We recommend reading through the detailed start below for users who have never used any HPC environment before.

For those who wish to get started quickly the basic steps:

1. Connect to the ASU VPN

2. Choose a connection method (Web Portal / SSH)

3. Transfer files as needed

4. Log in with your ASURITE & password

5. Run an interactive session or create an SBATCH script

• scheduler used on the ASU Supercomputers is called Slurm.

• Fairshare: Jobs will cost the user’s fairshare points to run, and the lower it is, the longer the job queuing time will be. Please spend it wisely. Here is more info about it.

Detailed Start

Connect through the Cisco VPN

All RC Research Computing resources require the user to be connected to the ASU Cisco AnyConnect VPN. While some areas of campus may permit connectivity without the VPN, the official stance is to use the VPN for all connections. This will help ensure the most friction-free experience. This is encouraged for all users, regardless if they are on campus or off campus.

Be sure to connect to sslvpn.asu.edu/2fa, and if . If prompted for a “second password,” provide either a DUO code, push to receive a DUO push request, phoneto authenticate via a phone call, or sms to authenticate via a text message.Please go to the SSL VPN page for

...

For additional details or to install the software, please navigate to this page.

...

...

Choosing a

...

Connection Method

Research Computing provides two methods for connecting to the supercomputer. Each has its their advantages and disadvantages.Our web portals have

1. Connecting to the Supercomputer with the Web Portal
   The web portal has become the standard for new users

...

1. . It provides a file system viewer and editor, a job submission tool, the ability to view the job queue, and a

...

1. zoo of interactive applications including a virtual desktop, Jupyter Lab, and RStudio. In the file manager, uploading files is as easy as dragging

...

1. and

...

1. dropping through the interface!

...

1. This web

...

1. portal is accessible through

...

1. sol

...

1. .asu.edu

...

1. .

   The virtual desktop

...

1. provided by sol.asu.edu is the best way to use graphical applications on the supercomputer. However, please try to avoid using graphical sessions unless you are first learning how to work with the supercomputer or you’re working with software that is only accessible through a graphical user interface. The goal of any interactive session on the supercomputer should be to develop a working

...

1. scheduling batch (SBATCH) script so that you may properly begin to take advantage of what supercomputing offers.
   

2. Connecting to

...

1. the Supercomputer with SSH
   SSH is the most versatile method

...

1. . It is ideal for submitting jobs at scale by allowing you to create custom workflows, submit multiple jobs simultaneously through job arrays, and explore options to avoid data loss through dependencies. However, it tends to be slower with interactive graphical applications

...

1. . If you intend to use MATLAB graphically (as opposed to MATLAB command line only) the screen draw will

...

1. be very slow. For graphical applications, we recommend our web portal instead.

Transfer files if needed

This is optional, however, most research is likely to require data sets to be imported. For details please see Transferring Files to and from the Supercomputer or /wiki/spaces/RC/pages/201490433

Log in to the supercomputer

...

Login to the Supercomputer

You are now ready to reach the login node! The login node is intended as a launching point to allocate compute nodes for your job. You only need to provide your ASURITE and password, if prompted.

Transfer Needed Files

This is optional. However, most research requires data sets or other files to be imported. For details, please see these tutorials on Transferring Files to and from the Supercomputer or using Google Drive & Globus.

Run an Interactive Session or Create an SBATCH Script

If you are using RC’s Jupyter or RStudio an interactive app provided in the web portal, this section can be skipped. If you are using a personally installed version of RStudio or Jupyter you should , please continue with reading this partsection.

Once you have a command prompt, there are two ways to get to a compute node:There are three ways to use resources on the supercomputer:

1. Creating an interactive session in the web portal using an interactive app, such as Jupyter, RStudio, or MATLAB. This will assign a compute node to your interactive session in an interactive app of your choice. This is a great option for users to become familiar with using the supercomputer as well as to develop, test, and debug code.
   

2. Starting an Interactive Session

...

1. in the shell. This will assign a compute node and connect your command prompt to it. This is good when working by hand to establish the commands needed to run your work. When your session disconnects, the interactive session also closes. Any unsaved work will be lost.
   

...

1. Scheduling Batch Scripts (Example). This is a method of telling the scheduler you want an unattended (or non-interactive) job to run. When

...

1. a sbatch script is submitted, the job will run until it either completes, fails, or runs out of time.

...

1. These sbatch scripts can be submitted through the shell or through the “Job Composer“ in the web portal.

Working with Python

Using Python on supercomputers is a little bit different than on workstations or local computers. Please use the system-provided mamba instead of conda or pip. Please follow our guide closely for the best practices with Python on the ASU supercomputers: Working with Python

Working with software

The system-wide installed software applications that are open to all the users, are often presented in the form of software modules or containers on supercomputers. Please follow this guide for how to find and use software: Using Software Modules

Recommended Reading

The page linked above will cover additional details on the supercomputer environment which will assist in maximizing your results.

Helpful Tips

...

• If your job is failing, a

...

• Job ID helps us significantly as we can pull detailed information about the job

...

• .

• If you are new to Linux, or need a refresher, Research Computing has created a guide at The Linux Shell on the Sol Supercomputer. For a great reference on building proficiency with command-line tools, we provide the following MIT link from CSAIL.

Additional Help

If you require further assistance on this topic, please don't hesitate to contact the Research Computing Team. To create a support ticket, kindly send an email to rtshelp@asu.edu. For quick inquiries, you're welcome to reach out via our #rc-support Slack Channel or attend our office hours for live assistance

We also offer a series of workshops. More information here: Educational Opportunities and Workshops