The ASU Supercomputers offers a python-environment manager, mamba, which can help alleviate these issues by keeping python packages properly contained.
This page describes the inner-workings of installing packages via various python methods and the impacts it may have.
The most common cause of Jupyter stopping working with “Bad State” is conflicts with user-installed python packages. See farther below for the fix.
Differences between installation methods
Environment Activated? | Installer | Command | Default Install Location | Note |
|---|---|---|---|---|
No mamba/no env | pip | pip install package_name | /usr/local/lib/pythonX.X/site-packages | This location is not user-writable, so python may default to installing to your $HOME. Avoid this route. |
pip (with --user) | pip (with --user) | pip install --user package_name | ~/.local/lib/pythonX.X/site-packages | This location is user-writable, but installs packages in a location Jupyter Notebook/Lab uses on the Web Portal. Avoid this route. |
source activate YYYY | mamba | mamba install package_name | ~/.conda/envs/<envname>/lib/pythonX.X/site-packages | This location is user-writable and will be installed properly via mambas safeguards. PREFERRED route. |
source activate YYYY | pip | pip install package_name | ~/.conda/envs/<envname>/lib/pythonX.X/site-packages | This location is user-writable and will be installed properly via mambas safeguards. This is an ACCEPTABLE route for packages that cannot be installed via mamba. |
source activate YYYY | pip (with --user) | pip install --user package_name | ~/.local/lib/pythonX.X/site-packages | This location is user-writable, but installs packages in a location Jupyter Notebook/Lab uses on the Web Portal. Avoid this route. |
MAIN TAKEAWAYS:
Prefer
mamba installeverywhere, where possible.Never use
pip --user installOnly use
pip installafter source activating your environment.
Fixing Jupyter Notebook in a “Bad State”
Fixing Jupyter Notebook that is having trouble starting or reporting “Bad State” can be done by removing (or renaming) the packages located in your ~/.local/lib directory.
By removing files that are universally installed to your $HOME directory, both Jupyter and conda environments will no longer have access to these files.
For packages installed here, they should be installed directly into the environment using the guidance above.
[asurite@login01:~]$ cd ~/.local/lib/ [asurite@login01:~/.local/lib]$ ll total 80 drwxr-x--- 3 asurite grp_usergrp 31 Jan 13 12:20 python3.11/ [asurite@login01:~/.local/lib]$ mv python3.11/ python3.11.old [asurite@login01:~/.local/lib]$ ll total 80 drwxr-x--- 3 asurite grp_usergrp 31 Jan 13 12:20 python3.11.old/
Go to the $HOME package installation dir:
cd ~/.local/lib/Identify any python packages directory--you may see multiple versions here, e.g.,
python3.11orpython2.7Rename the directories to have a new suffix, making them not get loaded automatically, e.g.,
.oldor.bak.Install these packages to a mamba environment using the table above.
Technical details
Background about Python Libraries Locations
When running python , python2, or python3, additional libraries are identified and python attempts to load these into the current session from /home/<asurite>/.local/lib/python<version>/site-libraries.
This is the default behavior of python, and in many cases for workstations, a dependable and time-saving functionality.
However, this default behavior is potentially disruptive when using Jupyter, which fully contains all the necessary packages in a curated, version-locked manner.
Background about Mamba-contained libraries
When using mamba create and mamba install , libraries are placed in to /home/<asurite>/.conda/envs/<envname>. This means you can have multiple environments that are completely independent of eachother which maybe both include the same packages, but do not interfere with eachother and can have different (and potentially conflicting) versions.
The selection of the environment is done with source activate and this ensures all mamba installand pip install remain safely contained within.