Skip to content

Conversation

@devreal
Copy link
Member

@devreal devreal commented May 19, 2017

This PR adds implicit local consistency to read operations following asynchronous writes, i.e., a dart_get or dart_get_blocking will read any value that was previously written by dart_put even if there has not been an explicit flush issued by the user. It does so by tracking the state of the segment and triggering a flush operation if a previous put operation to the same segment has not been flushed, i.e., if the segment is dirty. A read on a different segment will not trigger a flush.

The goal is to use this as the basis for operations on .async on DASH containers, see #371. In particular, the open problem of blocking reads following asynchronous writes will be solved while still allowing multiple put operations to be issued without requiring a flush.

To get there, this PR also restructures and unifies the handling of the local allocation window (segment with ID 0): it is now treated as a regular segment of DART_TEAM_ALL (which is the team owning this segment). Most notably, this gets rid of the special handling in DART communication operations, thus simplifying the code (at the expense of slightly higher overhead for access to local allocation segments). Some additional cleanup to the code has been done (hopefully making it more readable).

Background
MPI does not provide any guarantee on the ordering of asynchronous RMA operations and hence we have to make sure that all previous puts have finished before issuing a get to the same segment. This is in line with consistency guarantees provided by other PGAS approaches, e.g., CoArray. This will enable us to use STL algorithms on array.async. Since normal container accesses are blocking by default, they are not affected (we might consider changing the default at some point, though.)

Alternative solution:

  1. Have AsyncGlobRef call back into the container to mark it dirty (boo!). Also, that would not work if the user holds multiple references to the same value.
  2. Employ a caching mechanism to track individual asynchronous write accesses. That would be an overhead nightmare and won't scale.

@fuchsto
Copy link
Member

fuchsto commented May 20, 2017

I'm on it.
Can we arrange a Skype session tomorrow (Saturday) or on Monday?
The consistency guarantees seem overly strict to me, but I might just need some additional clarification.

@devreal
Copy link
Member Author

devreal commented Jul 14, 2017

I'm closing this PR because it has developed into a collection of random tweaks. I will disseminate the changes in smaller PRs...

@devreal devreal closed this Jul 14, 2017
@devreal devreal mentioned this pull request Oct 13, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants