Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

faqs: what versions of OpenMPI work with Flux? #94

Merged
merged 3 commits into from
Apr 2, 2021
Merged

faqs: what versions of OpenMPI work with Flux? #94

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
75f2c49
faqs: what versions of OpenMPI work with Flux?
garlick Apr 1, 2021
3b69895
faqs: add specifics about broken OpenMPI versions
SteVwonder Apr 2, 2021
c76816e
spell: add `backported` to the dictionary
SteVwonder Apr 2, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions faqs.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,3 +115,19 @@ The interesting part of the versioning comes from the multi-repo structure. Flux
A 'flux' meta-package (such as in spack or distro package managers) that would pull in compatible versions of the various sub-packages/repos is also versioned independently of any of its subcomponents. It is a similar situation for the flux-docs repo and the documentation up on readthedocs. Each repo has it's own documentation and that gets tagged and released along with the code, but the high-level "meta" documentation has it's own versioning that is divorced from any particular sub-packages/repos versioning.

.. TODO: we should make a table and put it in the docs too
----------------------------------------
What versions of OpenMPI work with Flux?
----------------------------------------

Flux plugins were added to OpenMPI 3.0.0. Generally, these plugins enable OpenMPI major versions 3 and 4 to work with Flux. OpenMPI must be configured with the Flux plugins enabled. Your installed version may be checked with:

.. code-block:: console
$ ompi_info|grep flux
MCA pmix: flux (MCA v2.1.0, API v2.0.0, Component v4.0.3)
MCA schizo: flux (MCA v2.1.0, API v1.0.0, Component v4.0.3)
Unfortunately, `a bug <https://github.com/open-mpi/ompi/issues/6730>`_ broke the Flux plugins in versions 3.1.0 through 4.0.1 (inclusive). The `trivial fix <https://github.com/open-mpi/ompi/pull/6764/commits/d4070d5f58f0c65aef89eea5910b202b8402e48b>`_ may be back-ported to the broken versions, if needed.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like that fix was already backported by Ralph and the OpenMPI team for 3.0.5, 3.1.5, and 4.0.2. So I think we should call out that 3.0.X , 3.1.X, and 4.0.Y+ ((where X>=5 and Y>=2) should work.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side-note: while looking up those PR numbers, I ran across a different issue that I was previously unaware of. It looks like a segfault in MPI_Finalize when using the Flux component was fixed in 4.0.6 and 4.1.1.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oof. Did you want to just push some changes directly to this branch? I'm all done for today, and I'm off tomorrow. o/w I'll pick it up on Tues.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure! Just pushed. I think that means @grondo, @cmoussa1, or @dongahn needs to review (if they have a chance).


The OpenMPI project dropped the Flux plugins--in fact, that whole abstraction layer that contains them--in the development branch that will become major version 5. There are two solutions being discussed, neither of which is implemented yet: `Running PRRTE inside a Flux allocation <https://github.com/flux-framework/flux-core/issues/3539>`_ and `Implementing a PMIx job shell plugin <https://github.com/flux-framework/flux-core/issues/3536>`_.