1 Overview
2 Identifying Large Files and Directories
3 Common Space-Consuming Applications and Solutions
- 3.1 Mamba
- 3.2 pip
- 3.3 Hugging Face
4 Additional Help

Overview

Home directories on ASU supercomputers have a 100 GiB limit, and unfortunately, there’s no option to expand them. For additional storage, we recommend using scratch or project storage.

If your home directory becomes full, you’ll encounter errors such as:

No space left on devices

When this happens, you may also be unable to create new sessions from the web portal. However, SSH access is not affected by home directory capacity, and all the commands in this guide assume you are working via SSH. For help connecting with SSH, refer to our Connecting to the Supercomputers with SSH guide.

When this happens, you’ll need to either delete unnecessary files or move them to a different storage system. Here’s how to check for space usage and address common issues.

Identifying Large Files and Directories

To see which files or directories are taking up space, use the command:

gdu ~

gdu (Go Disk Usage) is a fast and user-friendly tool for analyzing disk usage. Once in gdu:

Use the arrow keys to navigate up and down the list or into directories.
Press d to delete a file or directory directly from the interface.
Press q to quit

gdu --help
Pretty fast disk usage analyzer written in Go.

Usage:
  gdu [directory_to_scan] [flags]

Flags:
      --config-file string            Read config from file (default is $HOME/.gdu.yaml)
  -g, --const-gc                      Enable memory garbage collection during analysis with constant level set by GOGC
      --enable-profiling              Enable collection of profiling data and provide it on http://localhost:6060/debug/pprof/
  -L, --follow-symlinks               Follow symlinks for files, i.e. show the size of the file to which symlink points to (symlinks to directories are not followed)
  -h, --help                          help for gdu
  -i, --ignore-dirs strings           Absolute paths to ignore (separated by comma) (default [/proc,/dev,/sys,/run])
  -I, --ignore-dirs-pattern strings   Absolute path patterns to ignore (separated by comma)
  -X, --ignore-from string            Read absolute path patterns to ignore from file
  -f, --input-file string             Import analysis from JSON file
  -l, --log-file string               Path to a logfile (default "/dev/null")
  -m, --max-cores int                 Set max cores that GDU will use. 8 cores available (default 8)
  -c, --no-color                      Do not use colorized output
  -x, --no-cross                      Do not cross filesystem boundaries
      --no-delete                     Do not allow deletions
  -H, --no-hidden                     Ignore hidden directories (beginning with dot)
      --no-mouse                      Do not use mouse
      --no-prefix                     Show sizes as raw numbers without any prefixes (SI or binary) in non-interactive mode
  -p, --no-progress                   Do not show progress in non-interactive mode
  -u, --no-unicode                    Do not use Unicode symbols (for size bar)
  -n, --non-interactive               Do not run in interactive mode
  -o, --output-file string            Export all info into file as JSON
  -r, --read-from-storage             Read analysis data from persistent key-value storage
      --sequential                    Use sequential scanning (intended for rotating HDDs)
  -a, --show-apparent-size            Show apparent size
  -d, --show-disks                    Show all mounted disks
  -C, --show-item-count               Show number of items in directory
  -M, --show-mtime                    Show latest mtime of items in directory
  -B, --show-relative-size            Show relative size
      --si                            Show sizes with decimal SI prefixes (kB, MB, GB) instead of binary prefixes (KiB, MiB, GiB)
      --storage-path string           Path to persistent key-value storage directory (default is /tmp/badger) (default "/tmp/badger")
  -s, --summarize                     Show only a total in non-interactive mode
      --use-storage                   Use persistent key-value storage for analysis data (experimental)
  -v, --version                       Print version
      --write-config                  Write current configuration to file (default is $HOME/.gdu.yaml)

For more details on gdu, visit its GitHub page.

Alternatively, you can use:

ncdu ~

ncdu (ncurses Disk Usage) offers similar functionality, though it may run a bit slower.

Common Space-Consuming Applications and Solutions

Mamba

mamba, the Python package manager, stores temporary files, cached tarballs, and environments in your home directory. To free up space:

Remove any unused environments:
Clean up unnecessary packages, tarballs, and cache:

pip

We recommend using mamba over pip for Python package management whenever possible. There are lots of large files in ~/.local/lib that can be cleaned. To find these files, please check Python Package Installation Comparison.

However, if you must use pip, it often creates cache files in ~/.cache/pip. These files can be safely removed with:

Hugging Face

Hugging Face libraries frequently download large models to your home directory, storing them in ~/.cache/huggingface by default. To free up space and relocate these files to scratch storage:

Move the cache directory:
Update your shell to reflect this change:
Reload your shell:

We have lots of huggingface models already downloaded and ready to use, please check this page about how to find them: RC Community Datasets

Research Computing

Home Directory Cleanup Guide

Analytics

Overview

Identifying Large Files and Directories

Common Space-Consuming Applications and Solutions

Mamba

pip

Hugging Face

Additional Help

Related content