Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Marginal gains in send/recv fast-path #104

Merged
merged 3 commits into from
Jun 8, 2021
Merged

Marginal gains in send/recv fast-path #104

merged 3 commits into from
Jun 8, 2021

Conversation

csegarragonz
Copy link
Collaborator

@csegarragonz csegarragonz commented Jun 7, 2021

In this PR I revist the send/recv fast-path and try to prune unnecessary complexity. The most notable changes are:

  1. Switch from if-based undefined behaviour to assertions. This is risky and controversial, as the checks will be removed in Release builds. I say we use with care, but feels acceptable in this situation.
  2. Switch from lock-protected maps to lock-free vectors by initialising resources in advance, rather than lazily. Resource initialisation in both cases is thread safe as it is already protected by the locking in the world registry.

In my rudimentary stress test, I can see a noticeable increase in throughput (5-10%).

Lastly, removing the if+exception scheme, some tests which relied on REQUIRE_THROWS are now failing. It remains to be discussed what to do with them.

@csegarragonz csegarragonz added the mpi Related to the MPI implementation label Jun 7, 2021
@csegarragonz csegarragonz self-assigned this Jun 7, 2021
@@ -1227,21 +1213,6 @@ long MpiWorld::getLocalQueueSize(int sendRank, int recvRank)
return queue->size();
}

void MpiWorld::checkRankOnThisHost(int rank)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if we decide to keep the if statements, we can delete this method and keep the check in getLocalQueue through getHostForRank.

@csegarragonz csegarragonz force-pushed the fast-path branch 2 times, most recently from 89b48bd to 740ea9a Compare June 7, 2021 15:58
@csegarragonz csegarragonz marked this pull request as ready for review June 8, 2021 07:06
tests/test/scheduler/test_mpi_world.cpp Outdated Show resolved Hide resolved
include/faabric/scheduler/MpiWorld.h Outdated Show resolved Hide resolved
include/faabric/scheduler/MpiWorld.h Outdated Show resolved Hide resolved
src/scheduler/MpiWorld.cpp Outdated Show resolved Hide resolved
include/faabric/scheduler/MpiWorld.h Outdated Show resolved Hide resolved
@csegarragonz csegarragonz merged commit 1ac2b1e into master Jun 8, 2021
@csegarragonz csegarragonz deleted the fast-path branch June 8, 2021 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
mpi Related to the MPI implementation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants