/
MPI With Python (mpi4py)

MPI With Python (mpi4py)

Overview

To use mpi4py effectively on the supercomputers, you must align your Python environment with the system MPI

Liking mpi4py to an external MPI package ensures compatibility and performance when running parallel Python applications. However, this is not default behavior - mamba must be told explicitly to do this.

When to Use This

Use this method when:

  • You're writing or running Python code that uses mpi4py for parallel processing.

  • Your code or dependencies (like h5py) require MPI support.

  • You want to ensure compatibility with the system-provided OpenMPI version.

Building an Environment with mpi4py

Example for Sol with OpenMPI

module load mamba/latest openmpi/4.1.5 mamba create -n myenv -c conda-forge python=3 h5py=*=mpi_openmpi* mpi4py openmpi=4.1.5=external_*

Key Notes:

  • The openmpi=4.1.5=external_* spec ensures mamba uses the system's OpenMPI rather than installing its own.

  • h5py=*=mpi_openmpi* links h5py with MPI support.

  • Replace myenv with your preferred environment name.

 

Further Reading

https://conda-forge.org/docs/user/tipsandtricks/#using-external-message-passing-interface-mpi-libraries

Additional Help

We also offer a series of Educational Opportunities and Workshops.