Naming: To include MPI in the name or not

[cwpearson]

If you have a strong opinion, I also expect you have an argument to back it up 😄

[hmt23]

This topic was discussed here #3 a couple of weeks ago. I added a link for those not here when the discussion happened. I do not have strong opinions against including MPI in the name, but I would prefer without it.

[devreal]

The question is how strongly the semantics are tied to MPI and whether you can provide the exact same semantics with a different backend (say UCX/LCI/whathaveyou). Comm is very generic and I could make the argument that SHMEM (covered by kokkos-remote-space contains a form of (interprocess) communication. I think you are providing some form of message passing, which in turn is one form of communication. If you want to avoid MPI, maybe you could name it kokkos-messages?

I don't have strong feelings but I'm a bit concerned about the generality of kokkos-comm vs the strong ties to MPI semantics.

[cedricchevalier19]

I think neither of the names are very good. I agree that Comm is too vague, but I was afraid that MPI in the name was too tight with MPI standard and we do not want to provide all MPI functions and semantics.

However, I think @janciesko made the point earlier, Message Passing Interface is what we want to do, so perhaps MPI in the name is not so bad. But we have to be clear in the README that the first goal is a light wrapper over some MPI functionalities.

[cwpearson]

We're starting to pick up some momentum here with a variety of contributors and interest, so let's settle around a name.

I believe we should rename to Kokkos MPI/kokkos-mpi and stick with that name. This is the opposite of where we ended up at #3, which I think was wrongly decided because it emphasized a speculative future over the current reality.

I believe the core thrust of the project, as well as the semantics, in the near-to-intermediate term, will be defined by those of MPI. In support:

1. Our APIs so far are heavily inspired by MPI
2. Other backends (serial, NCCL) under near-term consideration are basically MPI-like
3. In practice apps really just use MPI and its semantics
4. TeuchosComm in Trilinos was named to accommodate non-MPI paradigms, and 20 years later...it's only MPI (I believe this is probably our future too).
5. No proposals for any non-MPI-like things have really had much public discussion or initial implementations.
6. We can still experiment with how MPI-like communication can be integrated with modern C++.

If enough interest or momentum for non-MPI-like things manifests, another project could be created (or this one could be used). Additionally, Kokkos Remote Spaces covers some non-MPI semantics.

Some arguments against that I've seen, and my refutations for those arguments

• "MPI" may give the impression we implement MPI, or implement more of MPI than we do
  • I don't think anyone familiar with Kokkos expects some kind of kokkos-based MPI implementation.
  • People will need to check with us and consult our docs before jumping in regardless
• We may want a semantics different than MPI
  • I have yet to hear a public proposal for this, or anyone who might use it. I don't want to choose a name around a speculative future.
• "MPI" suggests we're restricted to MPI, and not e.g. NCCL/UCX, whatever else
  • I think a user would understand they should reach for our library whever Kokkos + MPI meet, and even if we do some MPI-like other backends, that doesn't change.
  • Just as Kokkos Kernels has started to branch out into various solvers, being called Kokkos MPI doesn't prevent us from moving a bit away from pure MPI in the future as needed.

Overall, "Kokkos MPI" gives a more accurate, pithy flavor of what we're doing now, and what's been concretely proposed.

[hmt23]

Here is why I am personnaly against including MPI in the name.

I believe it is a mistake to begin implementing only a mpi wrapper without thinking about the (speculative) future. All decisions we will have now will be here for years when application users begin to use the API. I would prefer to discuss which semantics do we want to support with Kokkos instead of relying on MPI semantics, especially if we're going to consider several backend (NCCL, ROCCL, ONECCL, Gloo, MPI, RPC, Portals4, OFI, UCX and more, let's speculate even more, we need to speculate to not handle only near future).

My question is: do we really want to rely on MPI semantics? If we rename this project Kokkos-MPI, users will think we will respect the MPI semantics. My concern is that some of them are not well defined, especially in MPI+X context. For now, let's consider that MPI processes are Unix processes like in Open MPI and MPICH, even if it is not always the case (MPC-MPI, A-MPI).

If X is Kokkos, and we are on a Kokkos::parallel_for with OpenMP execution policy. The message ordering of our current implementation is ambigous and not defined at all. The first rule of point to point communication semantics (Order), chapter 3.5 on mpi-4.1 (page 55), does not apply to muti-threaded context. So if someone sends two messages to the same MPI process with the same tag and communicator (and it is what happen on our current implementation), the first send can match the first recv OR the second one. It is ambiguous and MPI forum members know that (see advise to user). Do we really want to rely on such semantics at the Kokkos API level for users?

Is it better to define our Kokkos semantics for inter-node communication? and choose KokkosComm as a name to avoid confusion?

Furthermore,

1. Yes but it is still our first implementation and semantics are not yet discussed.
2. Low level communication layers are not MPI-like.
3. In practice apps only cares about physics.
4. I hope no, but if we build KokkosComm as a MPI wrapper only, probably yes.
5. I agree and apologies for that. We are still working on it with Gabriel and Cédric.

As Christian said, it is not an "over-my-dead-body" issue.

[crtrott]

I totally acknowledge naming is hard but all in all I feel that a clearer scope in the name would be of benefit. The idea that KokkosMPI (kokkos-mpi) wraps MPI would likely not be to foreign to folks. In the end Kokkos Core itself in some sense wraps stuff underneat, KokkosFFT wraps fft libraries, KokkosKernels wraps all kinds of math libraries. I also think that even with that name it would be natural for it to still leverage stuff like NCCL which are anyway designed to be used in programs which otherwise may use MPI. We can set up a formal poll next week, and let that be the last word on this.

Note KokkosComm as name is not an "over-my-dead-body" issue.

[mhoemmen]

Here's a paraphrase of what I wrote in Slack on 2024/06/16.

I don't have a stake in this, but I agree with Christian that if you only plan to wrap MPI and minor extensions of MPI (like NCCL and clones), then it's reasonable to put "MPI" in the name. More importantly, if you start by wrapping just MPI, then the interface will forever reflect MPI's semantics. "MPI's semantics" includes the following.

• Communicators have a fixed number of processes with fixed IDs, with which one can communicate using point-to-point messages.
• Each communicator defines a distinct domain with respect to message matching and ordering.
• Users can clone and split communicators, but creating new processes (e.g., MPI_Comm_spawn) is expensive or impossible (it's not PVM).
• Data structures can be distributed over a communicator; such data structures are built of local data on each process in the communicator, and users' code has arbitrary access to and control of the data structure's local data (it's not MapReduce).

In that sense, NCCL and its clones have MPI semantics. Does Kokkos plan to wrap PVM? How about classic MapReduce, where users have no control over data distribution? How about an asynchronous PGAS model, where the parallel distribution is not fixed, getting access to "local" data requires explicit action and possibly collective communication, and such access must happen in a defined scope?

[dssgabriel]

I agree that KokkosMP might be ambiguous. Alternatively, I think kokkos-messages/KokkosMsg was one of the initial naming propostitions we had?

[hmt23]

@devreal I am happy to see that you are not strongly for KokkosMPI. Maybe, you can change your vote :)

[hmt23]

@masterleinad it is close to OpenMP but better than KokkosMPI. Go for it :)

[devreal]

I am strongly against KokkosComm because it is too generic. I don't think KokkosMP is dangerously close to OpenMP (and the confusion with Open MPI, don't get me started...)

IMHO, KokkosMsg only covers part of what this project provides (messages), unless of course you really mean MPI. Would we introduce KokkosColl for collective operations then?

Really the "problem" is that we are wrapping (and partially extending) MPI semantics. Maybe not all of them, but a sizable subset.

[hmt23]

I agree with you that KokkosComm is not a good name. That's why I complain about voting "for" KokkosComm; I am only strongly against KokkosMPI because we will not rely on MPI semantics defined by the MPI Forum. KokkosMsg is ok to me too :)

[JBludau]

On the ambition question, maybe this table helps

Concept	Basis Transform (KokkosFFT)	Parallel Machine Model (KokkosCore)	Linear Algebra (KokkosKernels)	Inter-Process Communication (KokkosComm/KokkosIPC)	Message Passing Interface (KokkosMPI)
Building blocks	cft, dft, etc.	parallel_for,view,etc.	matrix,vector,solvers, etc.	tbd	send, recv, comm,etc.
Implementations	cufft,rocfft,etc.	cuda,hip,etc.	...	MPI,NCCL,etc.	MPICH,openmpi,etc.

Personal part:
That being said, if the project only focuses around MPI, I think KokkosMPI (or derivates) would be fine and the pain to think about the building blocks and the abstract concept goes away. But I personally think this would solve less problems for me as a user, especially concerning future-proofness.
Of course it could always be expanded with a new name later but then users would have to rewrite their stuff. Starting with a general KokkosComm(derivates) could only focus on MPI at the start but the building blocks for the user would need to be MPI agnostic. But for me as a user it would definitely be an effort I have more interest in

[JBludau]

Thanks for your contribution, but in my opinion, KokkosIPC is a lousy name for the project (worse than MPI or Comm) because, as a computer scientist, I immediately think it is related to Unix IPC mechanisms for shared memory.

hmm going by the wikipedia definition IPC is simply the superset of MPI that would fit the description. But I agree that people might have images popping up in their heads

[devreal]

Even if we only support MPI today, I don't see a good reason why we should close the door to supporting other backends and finding better abstractions in the future.

It's not the backend that matters, it's the semantics we offer. Clearly, the semantics are that of MPI (send/recv of data with tags, peers with ranks in fixed-size communicators, collective operations on communicators). We add some syntactic sugar on top of that and accommodate some advanced features (stream semantics fwiw) that will eventually land in MPI too. You cannot change these semantics later without making all your users unhappy. If you want to provide a different model it should be a different project. So "keeping the door open for other models" is irrelevant. Having "MPI" in the name does in no way preclude you from providing a different implementation of the MPI semantics.

Keep in mind, it's the Message Passing Interface, not the Message Passing Implementation ;)

[hmt23]

You cannot change these semantics later without making all your users unhappy.

This is one of the reason why I am against the KokkosMPI name.

[devreal]

You cannot change these semantics later without making all your users unhappy.

This is one of the reason why I am against the KokkosMPI name.

Can you elaborate? KokkosMPI is merely a reflection of the semantics this project provides. Changing those semantics is a big deal. So why would a name representative of the current semantics be problematic?

[dssgabriel]

So why would a name representative of the current semantics be problematic?

We don't know if the project will be able to uphold the same semantics once we add support for other communication backends. Can we guarantee today that calling MPI_Send vs ncclSend (or any comparable functions, really) will mean the same thing? i.e., it's ok to re-use the buffer when the call returns?
It's these kinds of MPI-specific semantics that we do not want to be bound to, hence why I would prefer any other name that is not KokkosMPI.