Make Datatypes First-Level Objects

[devreal]

Problem

With the introduction of Sessions we require datatypes to exist across session boundaries, including the destruction of the last session and the creation of a new one. However, datatype operators are not allowed to be called outside of sessions. That is inconsistent and makes datatypes effectively zombies that cannot be destroyed once the last session is gone. At the same time, datatypes don't hang off any other object (sessions, communicators, etc).

In an AtoI in Section 5.1.9 we state:

The optimizations chosen during MPI_TYPE_COMMIT may no longer be optimal if a
session (or the World Model) is initialized or finalized.

In essence: creating a session to create a datatype for another session (an argument brought forward in past discussions) is a futile exercise.

Proposal

Add the MPI datatype functions to Table 11.1 to allow applications to create and destroy datatypes outside of sessions or the initialization of the WPM.

Changes to the Text

TBD

Impact on Implementations

Datatypes become true first-level objects that may require access to global internal state. Implementations will may have to retain some global state for datatypes.

Impact on Users

Datatypes not requiring sessions for creation and destruction opens new opportunities: third-party libraries (and language-bindings) can create datatypes at any time, regardless of the state of the MPI library.

References and Pull Requests

TBD

[bosilca]

This proposal fails to deal with optional datatypes or with any aspect of the datatype commit/optimization requiring proper knowledge about the setup (accelerators availability, heterogeneity, ...). Moreover, allowing third-party libraries to build complex types outside the MPI guidance will force either to drop or to force supporting all datatypes, aka. including the currently operational ones or embolden them to make strong, and possibly incorrect assumptions, about the MPI implementation.

As weakly as defined today, the current status quo is something implementors can live with, the support needed from the MPI library to build, store and optimize the type is available between any set of init/fini (world or session).

This is another idea that sounds great, but unless properly understood will create more issues down the line.

[devreal]

Fair enough. At least MPI_Type_free should be available outside of sessions to be able to destroy datatypes once they outlived their session(s).