Web Portal Jobs close immediately

Symptoms:

You are submitting jobs in the webportal (e.g., rstudio, jupyter, matlab) that queue properly, but when they start, they immediately complete/end without being able to use it.
Jobs started from the webportal end within 60 seconds of starting

Root Cause:

User-installed python packages to the $HOME/.local/lib directory conflict with the web portal python environment. The most common culprit is pip install --user

How to Fix:

Ensure your user-specified python packages are contained within python environments, e.g., mamba / conda.
Clear out conflicting files in $HOME/.local/lib/pythonX.XX

Details:

We encourage use of mamba environments. Using these will help prevent cross-pollination of user-desired packages and the web portal.

Removing these files from the default pip installation directory and installing them instead into an environment will fix this issue.

How it happens:

$ find ~/.local/lib/ -maxdepth 1
/home/wdizon/.local/lib/

$ pip install --user helloworld
Collecting helloworld
  Downloading helloworld-0.1dev.zip (20 kB)
  Preparing metadata (setup.py) ... done
Collecting Pylons>=1.0 (from helloworld)
  ..... additional stuff omitted .....

$ find ~/.local/lib/ -maxdepth 1
/home/wdizon/.local/lib/
/home/wdizon/.local/lib/python3.11

Note how the /.local/lib directory is populated with new python3.11 packages. These packages will conflict with the webportal, as it runs python scripts once the job is allocated (started). The python conflicts lead the job to fail during it’s execution and thus the job dies before the user can utilize it.

Deleting Conflicting Packages

[user@c001:~]$ cd ~/.local/lib/
[user@c001:~/.local/lib]$ ls
python3.11/
[user@c001:~/.local/lib]$ rm -rf python3.11/
[user@c001:~/.local/lib]$ ls
[user@c001:~/.local/lib]$

Once this is cleared out, your web portal jobs will work again.

Creating a Python Environment

Follow the steps here: https://asurc.atlassian.net/wiki/spaces/RC/pages/1663827969

When the environment is source activated, using mamba install is the preferred package installation step.
If the package is unavailable, the second-best is pip install.
pip install --user is never recommended.