Page Comparison

This guide walks you through the process of converting a Jupyter Notebook into a Python
script and running it on Sol.
This is useful when you have a long-running, computationally-intensive script that you
want to run on Sol.

Step 1: Convert the Jupyter Notebook to a Python script

To convert a Jupyter notebook (e.g, example_notebook.ipynb) to a Python script on Sol, you can request a short Jupyter session and click File -> Save and Export Notebook As ... -> Executable script. After the script is downloaded to your local machine, you can upload it back to Sol
follow by going to https://sol.asu.edu and using the Files tab.

Alternatively, you can convert the notebook to a Python script using the terminal by following these steps:

1. Open a command line interface on Sol by navigating to sol.asu.edu on your browser and
   selecting the "Sol Shell Access" option from the "System" menu option or by SSHing into
   Sol using the command ssh <asurite>@sol.asu.edu.

2. Request a lightwork compute node by running the command aux-interactive.

3. Load the Jupyter module by running the command module load jupyter/latest.

4. Convert the Jupyter Notebook to a Python script by running the command

...

This will create a Python script named example_notebook.py in the same directory.
After this, you can exit the lightwork compute node by running the command exit.

...

Review the generated example_notebook.py script and make any necessary adjustments to
ensure compatibility with non-interactive execution. A few common adjustments include:

• Removing or modifying any interactive elements (e.g., widgets, plots, etc.).

• Ensuring any file paths, inputs, or outputs are explicitly defined and relative to the
  working directory where the script will run.

• If the script includes plotting or graphical outputs, consider saving them to files
  (e.g., PNG, PDF) instead of displaying them interactively.

• Removing any cells that are not necessary for the sbatch job.

• Adding necessary imports and environment setup at the beginning of the script.

...

To submit the Python script as an sbatch job, you need to create an sbatch script that
specifies the resources and the commands required to run the job.
Below is an example sbatch script (submit_job.sbatch).
Note that you must activate the appropriate Mamba environment before running the script.
All Jupyter kernels, public and user-created, have an associated Mamba environment.

...

• Make sure to save your work in the Jupyter Notebook before converting it to a Python
  script.

• Ensure that all necessary files and dependencies are available in the working directory
  where the sbatch job will run.

• You can test and troubleshoot the Python script using the debug QOS by running
  interactive -q debug -t 15.