If you are familiar with supercomputing on Sol or Agave/Phoenix, Aloe will feel quite similar. However, due to the security requirements and technological differences, there are several changes between general-use systems like Sol and HIPPA complement systems like Aloe.
Internet Access is Restricted
While Sol and Agave/Phoenix can access the internet the same way any ASU-owned device would, Aloe is part of a restricted network with a tightly controlled list of allowed websites, and most of the internet is blocked.
Partitions and QoS
While Sol has a variety of QoSs and portions for different types of jobs, Aloe uses a simplified Partition and QoS setup. Currently, only the general partition and normal QoS exist. These are automatically set as defaults and do not need to be explicitly stated in job submission scripts.
See Requesting Resources on Aloe for more info
There is no /scratch, only /tmp
While Sol and Agave/Phoenix have very large scratch filesystems for high IO computing, Aloe does not. Computing directly from your Home directory or project space is perfectly acceptable as these are designed to be used for computing in this environment. If you have a job that benefits from fast disk access, then you can use the per-job ephemeral storage located at /tmp for faster results.
See ##Ignore for now Using /tmp in Aloe for more information.
Private Slurm Data
Many aspects of the Slurm scheduler have been marked as private and are not viewable to users. For example, you may see the message: slurm_load_partitions: Access/permission denied when viewing slurm information, such as the squeue or myjobs commands. This is not an error; this is expected behavior. This message can be ignored as it does not limit you from viewing information about your jobs and only restricts viewing overall information about partitions, which has been intentionally restricted.
Non-Accessible Compute Nodes
You are not able to SSH directly to a compute node, even if you have a job running on it. You will receive the error: Access denied by pam_slurm_adopt: you have no active jobs on this node. This is expected behavior.
You are not able to see information about other user's jobs.
While these restrictions are in place, it is still important to keep job names free from any PHI, PII, or anything that could be related to private information
Job Notifications
Aloe cannot send email notifications about jobs, such as when they start, end, or fail. Instead, Aloe will send Slack messages directly to you. To enable this, add the following to your job script:
#SBATCH --mail-type=ALL
Web Portal
The Aloe web portal is more restricted than Sol or Agave/Phoenix. For example:
Downloading files is restricted to using Globus
Uploading files is discouraged. See using Globus
The job composer has been disabled
The Active Jobs will only display your jobs regardless of the filter
A status page is not available at this time
There is no way to submit support tickets via the web portal. Instead, email asre-support@asu.edu
Data Ingress and Egress
Sol and Agave/Phoenix have less restricted ways of transferring data, and allow for ease of sharing data between users in the same group.
In a HIPPA environment, data movement and sharing are governed by strict policies to ensure no breach of confidential information. Per section 5 of the acceptable use policy:
Sensitive data (e.g., ePHI, PII, HIPAA, other restricted forms of data) must remain inside the KESC environment unless explicit permission to transfer data has been granted by the PI and a member of KESC Management.
If data is being sent to a subcontractor or other agent, additional contractual documents such as
a Data Use Agreement or Business Associate Agreement.Data may only be transferred into or out of ASRE using approved methods such as Globus, SFTP, and similar authenticated, authorized, and encrypted methods.
Because of this, downloading data from the Web Portal has been restricted.
Please see /wiki/spaces/ASREPUBLIC/pages/1909063756 for acceptable data transfer methods
If you have questions about how to move your data in or out of KESC, please contact asre-support@asu.edu
If you suspect a data leak of PII leak has occurred, report it immediately to KESC staff!