Building Software on ASU Supercomputers
A High-Performance Computing (HPC) environment like the Sol and Phoenix supercomputer varies greatly from a desktop environment like a personal workstation. Building software to run on supercomputers add considerations in a few areas: architecture, interconnects, binary compatibility, and permissions, to name a few. These considerations make it more time-consuming and complicated to successfully build software.
Research Computing staff will make a best-effort attempt to install software upon request. Ultimately, it is the user’s responsibility to find a viable solution for their workflow whether by compiling the code manually, using others' pre-compiled binaries or containers, or identifying alternative software.
Some software is central to a HPC environment, such as compiler suites, Python, R, and their library dependencies; Research Computing admins will always install and make Environment Modules to allow quick and easy access to these highly-requested software programs.
In others cases, such as 1) highly domain-specific software, 2) largely-untested code, and 3) development versions of software, it may be imprudent to create public Environment modules. Research Computing admins can still assist by building this software in more personalized locations where they can be executed without the use of the “module load” system.
For software built for use by only one user, it can be installed to the user’s HOME
directory; software like this typically only needs to be invoked by the filepath to operate, whether interactively or in a job script.
For software usable by a group/lab, it can be installed to the group share directory, e.g., /data/grp_XXXX
where XXXX
is the user’s group (visible with the command groups
). Software installed in this location can freely be used by all users sharing that group, with both read and write permissions available. Source code that needs to be modified with custom changes will typically fall in this category.
For software requests that have wide appeal to many groups and users, this software will typically be built in the public fileshare, e.g., /packages/apps
and will be accompanied by a public environment modulefile which can be loaded with module load <package/version>
.
Requesting software to be built
Requests for software installations can be made in a support ticket.
Be sure to include all pertinent information:
software URL
version requirements
plugin requirements
location of downloaded source code (if behind paywall or login, or the filepath downloaded to)
The supercomputer is 100% Linux running Rocky Linux 8.x, a Redhat variant (RHEL). The supercomputer does not support Windows software.
Software explicitly requiring Debian, Ubuntu, etc., may compile and we will make a best-effort attempt to help build the software. Beyond that, seek a container solution, as other Linux distributions rely on software managers and libraries likely incompatible with RedHat-compatible Rocky Linux.
Compiling software from source
Luckily, many of the compilation considerations mentioned above can be handled automatically, with provided build scripts and makefiles from the software author. At the minimum, an understanding of Linux filesystem permissions is required to build software, as most software defaults to installing files in locations not permissible by regular, non-admin users.
Filesystem Permissions & Shared Storage
A valid--albeit abbreviated--installation of python might go like this:
wget https://www.python.org/ftp/python/3.9.7/Python-3.9.7.tgz
tar -xf Python-3.9.7.tgz
cd Python-3.9.7
./configure
make
make install
These steps will work all the way until make install
, when it attempts to install to /usr/local/python-3.9.7
and gets a Permission Denied
error.
This error is not an indication of a failed build, but simple a failure to copy the binaries to root
-owned paths on a node. Many users may recognize this as a sudo
-solvable problem, but in HPC environments, that is not the case.
sudo access
Sudo access is reserved explicitly for Research Computing admins. If a software installation attempts to install to a local directory and fails, the answer is not to request more permission to be able to install to a local directory. Instead redirect the installation to a shared storage space, like your HOME
directory or group shared storage.
If you did have sudo
access, you would be able to install software to the default location /usr/local/…
. However, when you disconnect and reconnect to the supercomputer, you can not reasonably expect to be on the exact same node, which means your build is inaccessible to your job. /home/[asurite]/
on the other hand is available on every node.
Changing build directories
The above installation could therefore be adjusted in a very simple way to install to your HOME
directory:
...
$ ./configure --prefix=/home/[asurite]/.local/opt/python-3.9.7
$ make
$ make install
Container solutions
Apptainer (aka Singularity) is available on the RC supercomputers. Many times, software is distributed in an easy-to-download package (.sif
file) which can be uploaded to your HOME
directory and used immediately.
In some cases where a pre-configured Singularity is not available, but one can be built with a recipe, the container must be built on your local workstation (rather than the supercomputer), due to sudo
/ root
requirements.
Please check this dedicated page for more information Apptainer usage .