Gaudi Accelerators
On the ASU Sol supercomputer, users have access to seven Gaudi nodes, each equipped with eight Gaudi acceleratorsfeaturing 92 GB of memory per device. These nodes are available in the Sol partition for general use.
Requesting Gaudis from terminal
On Sol, the most straightforward way to get access to the accelerators are with salloc:
salloc -p gaudi -c 152 --mem=0 -G 1Use the -c flag to specify the number of CPU cores, and --mem to set the desired system memory (up to --mem=515000M). Use -G 1 for number of accelerators.
Each Gaudi accelerator should be paired with at least one CPU core and 48 GB of memory to ensure efficient data transfer and prevent I/O bottlenecks. Because each node is allocated exclusively to a single user, any unused CPUs or memory will remain idle. Given that CPU and memory resources are relatively inexpensive compared to accelerator time, it’s generally best to prioritize maximizing throughput rather than minimizing resource usage.
Using Gaudis with Jupyter Notebook
By selecting the gaudi partition with the public QoS, you can request Gaudis for your Jupyter Notebook.
Once your job allocates, you can then select any of the Gaudi-specific Jupyter kernels:
gaudi-pytorch
gaudi-pytorch-diffusion (stable diffusion)
Checking Accelerator Usage
You can check the utilization of each accelerator with hl-smi:
[rcsparky@gaudi003:~]$ hl-smi
+-----------------------------------------------------------------------------+
| HL-SMI Version: hl-1.22.0-fw-61.3.2.0 |
| Driver Version: 1.22.0-5f8fa9f |
| Nic Driver Version: 1.22.0-5f8fa9f |
|-------------------------------+----------------------+----------------------+
| AIP Name Persistence-M| Bus-Id Disp.A | Volatile Uncor-Events|
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | AIP-Util Compute M. |
|===============================+======================+======================|
| 0 HL-225 N/A | 0000:b3:00.0 N/A | 0 |
| N/A 28C P0 87W / 600W | 768MiB / 98304MiB | 0% 0% |
|-------------------------------+----------------------+----------------------+
| 1 HL-225 N/A | 0000:b4:00.0 N/A | 0 |
| N/A 28C P0 81W / 600W | 768MiB / 98304MiB | 0% 0% |
|-------------------------------+----------------------+----------------------+
| 2 HL-225 N/A | 0000:cc:00.0 N/A | 0 |
| N/A 29C P0 82W / 600W | 768MiB / 98304MiB | 0% 0% |
|-------------------------------+----------------------+----------------------+
| 3 HL-225 N/A | 0000:cd:00.0 N/A | 0 |
| N/A 27C P0 79W / 600W | 768MiB / 98304MiB | 0% 0% |
|-------------------------------+----------------------+----------------------+
| 4 HL-225 N/A | 0000:19:00.0 N/A | 0 |
| N/A 27C P0 87W / 600W | 768MiB / 98304MiB | 0% 0% |
|-------------------------------+----------------------+----------------------+
| 5 HL-225 N/A | 0000:1a:00.0 N/A | 0 |
| N/A 30C P0 73W / 600W | 768MiB / 98304MiB | 0% 0% |
|-------------------------------+----------------------+----------------------+
| 6 HL-225 N/A | 0000:43:00.0 N/A | 0 |
| N/A 29C P0 68W / 600W | 768MiB / 98304MiB | 0% 0% |
|-------------------------------+----------------------+----------------------+
| 7 HL-225 N/A | 0000:44:00.0 N/A | 0 |
| N/A 27C P0 81W / 600W | 768MiB / 98304MiB | 0% 0% |
|-------------------------------+----------------------+----------------------+