-
Notifications
You must be signed in to change notification settings - Fork 28
PB-159: remove weights from gRPC messages #298
PB-159: remove weights from gRPC messages #298
Conversation
References: https://xainag.atlassian.net/browse/PB-159 Needs to be merged along with: - https://github.com/xainag/xain-proto/pull/25 - https://github.com/xainag/xain-sdk/pull/88 - #298 Summary: Remove the weights from the gRPC messages. From now on, weights will be exchanged via s3 buckets. The sequence diagram below illustrate this new behavior. At the beginning of a round (1) the selected participants send a `StartTrainingRound` request, and the coordinator response with the same `StartTrainingRoundResponse` that does not contain the global weights anymore. Instead, the participant fetches these weights from the store (2). S3 buckets are key-value stores, and the key for global weights is the round number. Then, the participant trains. Once done, it uploads its local weights to the S3 bucket (3). The key is `<participant_id>/<round_number>`. Finally (4), the participant sends it's `EndTrainingRequest`. Before answering, the coordinator retrieves the local weights the participant has uploaded. _**Important note**: At the moment, the participants don't know their ID, because the coordinator does send it to them. Thus, they currently generate a random ID when they start, and send it to the coordinator so that it can retrieve the participant's weights. This is why the `EndTrainingRoundRequest` currently has a `participant_id` field._ ``` P C Store 1. | StartTrainingRoundRequest | | | -----------------------------> | | | StartTrainingRoundResponse | | | <----------------------------- | | | | | | Get global weights (key="round") | 2. | ------------------------------------------------------> | | Global weights | | <------------------------------------------------------ | | | | | [train...] | | | | | 3. | Set local weights (key="participant/round") | | ------------------------------------------------------> | | Ok | | <------------------------------------------------------ | | | | 4. | EndTrainingRoundRequest | | | -----------------------------> | Get local weights (key="participant/round") | | ---------------------> | | | Local weights | | EndTrainingRoundResponse | <--------------------> | | <----------------------------- | | ``` At the end of the round, the coordinator writes the weights to the s3 bucket, using the next upcoming round number as key (see the sequence diagram below). ``` P C Store | EndTrainingRoundRequest | | | -----------------------------> | Get local weights (key="participant/round") | | ---------------------> | | | Local weights | | EndTrainingRoundResponse | <--------------------> | | <----------------------------- | | | | | | | Set global weights (key="round + 1") | | ---------------------> | | | Ok | | | <--------------------> | ``` Implementation notes: - Initially, we thought we would be using different buckets for the local and global weights. But for now, we use the same bucket for local and global weights for now - We currently store the global weights under different keys. It turns out that this brings un-necessary complexity so we'll probably simplify this in the future - For now, the coordinator doesn't send any storage information to the participants. Thus, the participants need to be configured with the storage information. In the future, the `StartTrainingRoundResponse` could contain the endpoint url, bucket name, etc.
218ecc2
to
4f17e1f
Compare
References: https://xainag.atlassian.net/browse/PB-159 Needs to be merged along with: - https://github.com/xainag/xain-proto/pull/25 - https://github.com/xainag/xain-sdk/pull/88 - #298 Summary: Remove the weights from the gRPC messages. From now on, weights will be exchanged via s3 buckets. The sequence diagram below illustrate this new behavior. At the beginning of a round (1) the selected participants send a `StartTrainingRound` request, and the coordinator response with the same `StartTrainingRoundResponse` that does not contain the global weights anymore. Instead, the participant fetches these weights from the store (2). S3 buckets are key-value stores, and the key for global weights is the round number. Then, the participant trains. Once done, it uploads its local weights to the S3 bucket (3). The key is `<participant_id>/<round_number>`. Finally (4), the participant sends it's `EndTrainingRequest`. Before answering, the coordinator retrieves the local weights the participant has uploaded. _**Important note**: At the moment, the participants don't know their ID, because the coordinator does send it to them. Thus, they currently generate a random ID when they start, and send it to the coordinator so that it can retrieve the participant's weights. This is why the `EndTrainingRoundRequest` currently has a `participant_id` field._ ``` P C Store 1. | StartTrainingRoundRequest | | | -----------------------------> | | | StartTrainingRoundResponse | | | <----------------------------- | | | | | | Get global weights (key="round") | 2. | ------------------------------------------------------> | | Global weights | | <------------------------------------------------------ | | | | | [train...] | | | | | 3. | Set local weights (key="participant/round") | | ------------------------------------------------------> | | Ok | | <------------------------------------------------------ | | | | 4. | EndTrainingRoundRequest | | | -----------------------------> | Get local weights (key="participant/round") | | ---------------------> | | | Local weights | | EndTrainingRoundResponse | <--------------------> | | <----------------------------- | | ``` At the end of the round, the coordinator writes the weights to the s3 bucket, using the next upcoming round number as key (see the sequence diagram below). ``` P C Store | EndTrainingRoundRequest | | | -----------------------------> | Get local weights (key="participant/round") | | ---------------------> | | | Local weights | | EndTrainingRoundResponse | <--------------------> | | <----------------------------- | | | | | | | Set global weights (key="round + 1") | | ---------------------> | | | Ok | | | <--------------------> | ``` Implementation notes: - Initially, we thought we would be using different buckets for the local and global weights. But for now, we use the same bucket for local and global weights for now - We currently store the global weights under different keys. It turns out that this brings un-necessary complexity so we'll probably simplify this in the future - For now, the coordinator doesn't send any storage information to the participants. Thus, the participants need to be configured with the storage information. In the future, the `StartTrainingRoundResponse` could contain the endpoint url, bucket name, etc.
4f17e1f
to
14304a4
Compare
References: https://xainag.atlassian.net/browse/PB-159 Needs to be merged along with: - https://github.com/xainag/xain-proto/pull/25 - https://github.com/xainag/xain-sdk/pull/88 - #298 Summary: Remove the weights from the gRPC messages. From now on, weights will be exchanged via s3 buckets. The sequence diagram below illustrate this new behavior. At the beginning of a round (1) the selected participants send a `StartTrainingRound` request, and the coordinator response with the same `StartTrainingRoundResponse` that does not contain the global weights anymore. Instead, the participant fetches these weights from the store (2). S3 buckets are key-value stores, and the key for global weights is the round number. Then, the participant trains. Once done, it uploads its local weights to the S3 bucket (3). The key is `<participant_id>/<round_number>`. Finally (4), the participant sends it's `EndTrainingRequest`. Before answering, the coordinator retrieves the local weights the participant has uploaded. _**Important note**: At the moment, the participants don't know their ID, because the coordinator does send it to them. Thus, they currently generate a random ID when they start, and send it to the coordinator so that it can retrieve the participant's weights. This is why the `EndTrainingRoundRequest` currently has a `participant_id` field._ ``` P C Store 1. | StartTrainingRoundRequest | | | -----------------------------> | | | StartTrainingRoundResponse | | | <----------------------------- | | | | | | Get global weights (key="round") | 2. | ------------------------------------------------------> | | Global weights | | <------------------------------------------------------ | | | | | [train...] | | | | | 3. | Set local weights (key="participant/round") | | ------------------------------------------------------> | | Ok | | <------------------------------------------------------ | | | | 4. | EndTrainingRoundRequest | | | -----------------------------> | Get local weights (key="participant/round") | | ---------------------> | | | Local weights | | EndTrainingRoundResponse | <--------------------> | | <----------------------------- | | ``` At the end of the round, the coordinator writes the weights to the s3 bucket, using the next upcoming round number as key (see the sequence diagram below). ``` P C Store | EndTrainingRoundRequest | | | -----------------------------> | Get local weights (key="participant/round") | | ---------------------> | | | Local weights | | EndTrainingRoundResponse | <--------------------> | | <----------------------------- | | | | | | | Set global weights (key="round + 1") | | ---------------------> | | | Ok | | | <--------------------> | ``` Implementation notes: - Initially, we thought we would be using different buckets for the local and global weights. But for now, we use the same bucket for local and global weights for now - We currently store the global weights under different keys. It turns out that this brings un-necessary complexity so we'll probably simplify this in the future - For now, the coordinator doesn't send any storage information to the participants. Thus, the participants need to be configured with the storage information. In the future, the `StartTrainingRoundResponse` could contain the endpoint url, bucket name, etc.
752a2f7
to
573dc12
Compare
References: https://xainag.atlassian.net/browse/PB-159 Needs to be merged along with: - https://github.com/xainag/xain-proto/pull/25 - https://github.com/xainag/xain-sdk/pull/88 - #298 Summary: Remove the weights from the gRPC messages. From now on, weights will be exchanged via s3 buckets. The sequence diagram below illustrate this new behavior. At the beginning of a round (1) the selected participants send a `StartTrainingRound` request, and the coordinator response with the same `StartTrainingRoundResponse` that does not contain the global weights anymore. Instead, the participant fetches these weights from the store (2). S3 buckets are key-value stores, and the key for global weights is the round number. Then, the participant trains. Once done, it uploads its local weights to the S3 bucket (3). The key is `<round_number>/<participant_id>`. Finally (4), the participant sends it's `EndTrainingRequest`. Before answering, the coordinator retrieves the local weights the participant has uploaded. _**Important note**: At the moment, the participants don't know their ID, because the coordinator does not send it to them. Thus, they currently generate a random ID when they start, and send it to the coordinator so that it can retrieve the participant's weights. This is why the `EndTrainingRoundRequest` currently has a `participant_id` field._ ``` P C Store 1. | StartTrainingRoundRequest | | | -----------------------------> | | | StartTrainingRoundResponse | | | <----------------------------- | | | | | | Get global weights (key="round/global") | 2. | ------------------------------------------------------> | | Global weights | | <------------------------------------------------------ | | | | | [train...] | | | | | 3. | Set local weights (key="round/participant") | | ------------------------------------------------------> | | Ok | | <------------------------------------------------------ | | | | 4. | EndTrainingRoundRequest | | | -----------------------------> | Get local weights (key="round/participant") | | ---------------------> | | | Local weights | | EndTrainingRoundResponse | <--------------------> | | <----------------------------- | | ``` At the end of the round, the coordinator writes the weights to the s3 bucket, using the next upcoming round number as key (see the sequence diagram below). ``` P C Store | EndTrainingRoundRequest | | | -----------------------------> | Get local weights (key="round/participant") | | ---------------------> | | | Local weights | | EndTrainingRoundResponse | <--------------------> | | <----------------------------- | | | | | | | Set global weights (key="round+1/participant") | | ---------------------> | | | Ok | | | <--------------------> | ``` Implementation notes: - Initially, we thought we would be using different buckets for the local and global weights. But for now, we use the same bucket for local and global weights for now - We currently store the global weights under different keys. It turns out that this brings un-necessary complexity so we'll probably simplify this in the future - For now, the coordinator doesn't send any storage information to the participants. Thus, the participants need to be configured with the storage information. In the future, the `StartTrainingRoundResponse` could contain the endpoint url, bucket name, etc. handle review comments
7dabc27
to
a114952
Compare
References: https://xainag.atlassian.net/browse/PB-159 Needs to be merged along with: - https://github.com/xainag/xain-proto/pull/25 - https://github.com/xainag/xain-sdk/pull/88 - #298 Summary: Remove the weights from the gRPC messages. From now on, weights will be exchanged via s3 buckets. The sequence diagram below illustrate this new behavior. At the beginning of a round (1) the selected participants send a `StartTrainingRound` request, and the coordinator response with the same `StartTrainingRoundResponse` that does not contain the global weights anymore. Instead, the participant fetches these weights from the store (2). S3 buckets are key-value stores, and the key for global weights is the round number. Then, the participant trains. Once done, it uploads its local weights to the S3 bucket (3). The key is `<round_number>/<participant_id>`. Finally (4), the participant sends it's `EndTrainingRequest`. Before answering, the coordinator retrieves the local weights the participant has uploaded. _**Important note**: At the moment, the participants don't know their ID, because the coordinator does not send it to them. Thus, they currently generate a random ID when they start, and send it to the coordinator so that it can retrieve the participant's weights. This is why the `EndTrainingRoundRequest` currently has a `participant_id` field._ ``` P C Store 1. | StartTrainingRoundRequest | | | -----------------------------> | | | StartTrainingRoundResponse | | | <----------------------------- | | | | | | Get global weights (key="round/global") | 2. | ------------------------------------------------------> | | Global weights | | <------------------------------------------------------ | | | | | [train...] | | | | | 3. | Set local weights (key="round/participant") | | ------------------------------------------------------> | | Ok | | <------------------------------------------------------ | | | | 4. | EndTrainingRoundRequest | | | -----------------------------> | Get local weights (key="round/participant") | | ---------------------> | | | Local weights | | EndTrainingRoundResponse | <--------------------> | | <----------------------------- | | ``` At the end of the round, the coordinator writes the weights to the s3 bucket, using the next upcoming round number as key (see the sequence diagram below). ``` P C Store | EndTrainingRoundRequest | | | -----------------------------> | Get local weights (key="round/participant") | | ---------------------> | | | Local weights | | EndTrainingRoundResponse | <--------------------> | | <----------------------------- | | | | | | | Set global weights (key="round+1/participant") | | ---------------------> | | | Ok | | | <--------------------> | ``` Implementation notes: - Initially, we thought we would be using different buckets for the local and global weights. But for now, we use the same bucket for local and global weights for now - We currently store the global weights under different keys. It turns out that this brings un-necessary complexity so we'll probably simplify this in the future - For now, the coordinator doesn't send any storage information to the participants. Thus, the participants need to be configured with the storage information. In the future, the `StartTrainingRoundResponse` could contain the endpoint url, bucket name, etc.
a114952
to
fddbf56
Compare
Please rebase this branch |
I'll probably wait a little because there are other PRs in the pipe that will create more conflicts. The PR is reviewable as it is anyway. |
References: https://xainag.atlassian.net/browse/PB-159 Needs to be merged along with: - https://github.com/xainag/xain-proto/pull/25 - https://github.com/xainag/xain-sdk/pull/88 - #298 Summary: Remove the weights from the gRPC messages. From now on, weights will be exchanged via s3 buckets. The sequence diagram below illustrate this new behavior. At the beginning of a round (1) the selected participants send a `StartTrainingRound` request, and the coordinator response with the same `StartTrainingRoundResponse` that does not contain the global weights anymore. Instead, the participant fetches these weights from the store (2). S3 buckets are key-value stores, and the key for global weights is the round number. Then, the participant trains. Once done, it uploads its local weights to the S3 bucket (3). The key is `<round_number>/<participant_id>`. Finally (4), the participant sends it's `EndTrainingRequest`. Before answering, the coordinator retrieves the local weights the participant has uploaded. _**Important note**: At the moment, the participants don't know their ID, because the coordinator does not send it to them. Thus, they currently generate a random ID when they start, and send it to the coordinator so that it can retrieve the participant's weights. This is why the `EndTrainingRoundRequest` currently has a `participant_id` field._ ``` P C Store 1. | StartTrainingRoundRequest | | | -----------------------------> | | | StartTrainingRoundResponse | | | <----------------------------- | | | | | | Get global weights (key="round/global") | 2. | ------------------------------------------------------> | | Global weights | | <------------------------------------------------------ | | | | | [train...] | | | | | 3. | Set local weights (key="round/participant") | | ------------------------------------------------------> | | Ok | | <------------------------------------------------------ | | | | 4. | EndTrainingRoundRequest | | | -----------------------------> | Get local weights (key="round/participant") | | ---------------------> | | | Local weights | | EndTrainingRoundResponse | <--------------------> | | <----------------------------- | | ``` At the end of the round, the coordinator writes the weights to the s3 bucket, using the next upcoming round number as key (see the sequence diagram below). ``` P C Store | EndTrainingRoundRequest | | | -----------------------------> | Get local weights (key="round/participant") | | ---------------------> | | | Local weights | | EndTrainingRoundResponse | <--------------------> | | <----------------------------- | | | | | | | Set global weights (key="round+1/participant") | | ---------------------> | | | Ok | | | <--------------------> | ``` Implementation notes: - Initially, we thought we would be using different buckets for the local and global weights. But for now, we use the same bucket for local and global weights for now - We currently store the global weights under different keys. It turns out that this brings un-necessary complexity so we'll probably simplify this in the future - For now, the coordinator doesn't send any storage information to the participants. Thus, the participants need to be configured with the storage information. In the future, the `StartTrainingRoundResponse` could contain the endpoint url, bucket name, etc.
fddbf56
to
1770099
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good, just a few minor issues / questions. on a more general note, it does not look to me too difficult to have direct weights transfer (the previous behaviour) as just a "special case" of the present one. But this can be a separate issue.
3ec2d98
References: https://xainag.atlassian.net/browse/PB-159 Needs to be merged along with: - https://github.com/xainag/xain-proto/pull/25 - https://github.com/xainag/xain-sdk/pull/88 - #298 Summary: Remove the weights from the gRPC messages. From now on, weights will be exchanged via s3 buckets. The sequence diagram below illustrate this new behavior. At the beginning of a round (1) the selected participants send a `StartTrainingRound` request, and the coordinator response with the same `StartTrainingRoundResponse` that does not contain the global weights anymore. Instead, the participant fetches these weights from the store (2). S3 buckets are key-value stores, and the key for global weights is the round number. Then, the participant trains. Once done, it uploads its local weights to the S3 bucket (3). The key is `<round_number>/<participant_id>`. Finally (4), the participant sends it's `EndTrainingRequest`. Before answering, the coordinator retrieves the local weights the participant has uploaded. _**Important note**: At the moment, the participants don't know their ID, because the coordinator does not send it to them. Thus, they currently generate a random ID when they start, and send it to the coordinator so that it can retrieve the participant's weights. This is why the `EndTrainingRoundRequest` currently has a `participant_id` field._ ``` P C Store 1. | StartTrainingRoundRequest | | | -----------------------------> | | | StartTrainingRoundResponse | | | <----------------------------- | | | | | | Get global weights (key="round/global") | 2. | ------------------------------------------------------> | | Global weights | | <------------------------------------------------------ | | | | | [train...] | | | | | 3. | Set local weights (key="round/participant") | | ------------------------------------------------------> | | Ok | | <------------------------------------------------------ | | | | 4. | EndTrainingRoundRequest | | | -----------------------------> | Get local weights (key="round/participant") | | ---------------------> | | | Local weights | | EndTrainingRoundResponse | <--------------------> | | <----------------------------- | | ``` At the end of the round, the coordinator writes the weights to the s3 bucket, using the next upcoming round number as key (see the sequence diagram below). ``` P C Store | EndTrainingRoundRequest | | | -----------------------------> | Get local weights (key="round/participant") | | ---------------------> | | | Local weights | | EndTrainingRoundResponse | <--------------------> | | <----------------------------- | | | | | | | Set global weights (key="round+1/participant") | | ---------------------> | | | Ok | | | <--------------------> | ``` Implementation notes: - Initially, we thought we would be using different buckets for the local and global weights. But for now, we use the same bucket for local and global weights for now - We currently store the global weights under different keys. It turns out that this brings un-necessary complexity so we'll probably simplify this in the future - For now, the coordinator doesn't send any storage information to the participants. Thus, the participants need to be configured with the storage information. In the future, the `StartTrainingRoundResponse` could contain the endpoint url, bucket name, etc.
Co-Authored-By: kwok <kwok.cheung@xain.io>
Co-Authored-By: kwok <kwok.cheung@xain.io>
Co-Authored-By: kwok <kwok.cheung@xain.io>
Co-Authored-By: kwok <kwok.cheung@xain.io>
952a0ee
to
bb61423
Compare
I addressed the reviews and rebased. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
all good, thanks.
Regarding the changes in |
Ah, I just saw your message on Slack already covering this. 🥇 |
* PB-159: update xain-{proto,sdk} dependencies to the right branch Follow-up of #298 * PB-159: fix broken tests This test broke due to a change in xain-sdk. Ref: https://github.com/xainag/xain-sdk/pull/91/
* PB-159: remove weights from gRPC messages References: https://xainag.atlassian.net/browse/PB-159 Needs to be merged along with: - https://github.com/xainag/xain-proto/pull/25 - https://github.com/xainag/xain-sdk/pull/88 - #298 Summary: Remove the weights from the gRPC messages. From now on, weights will be exchanged via s3 buckets. The sequence diagram below illustrate this new behavior. At the beginning of a round (1) the selected participants send a `StartTrainingRound` request, and the coordinator response with the same `StartTrainingRoundResponse` that does not contain the global weights anymore. Instead, the participant fetches these weights from the store (2). S3 buckets are key-value stores, and the key for global weights is the round number. Then, the participant trains. Once done, it uploads its local weights to the S3 bucket (3). The key is `<round_number>/<participant_id>`. Finally (4), the participant sends it's `EndTrainingRequest`. Before answering, the coordinator retrieves the local weights the participant has uploaded. _**Important note**: At the moment, the participants don't know their ID, because the coordinator does not send it to them. Thus, they currently generate a random ID when they start, and send it to the coordinator so that it can retrieve the participant's weights. This is why the `EndTrainingRoundRequest` currently has a `participant_id` field._ ``` P C Store 1. | StartTrainingRoundRequest | | | -----------------------------> | | | StartTrainingRoundResponse | | | <----------------------------- | | | | | | Get global weights (key="round/global") | 2. | ------------------------------------------------------> | | Global weights | | <------------------------------------------------------ | | | | | [train...] | | | | | 3. | Set local weights (key="round/participant") | | ------------------------------------------------------> | | Ok | | <------------------------------------------------------ | | | | 4. | EndTrainingRoundRequest | | | -----------------------------> | Get local weights (key="round/participant") | | ---------------------> | | | Local weights | | EndTrainingRoundResponse | <--------------------> | | <----------------------------- | | ``` At the end of the round, the coordinator writes the weights to the s3 bucket, using the next upcoming round number as key (see the sequence diagram below). ``` P C Store | EndTrainingRoundRequest | | | -----------------------------> | Get local weights (key="round/participant") | | ---------------------> | | | Local weights | | EndTrainingRoundResponse | <--------------------> | | <----------------------------- | | | | | | | Set global weights (key="round+1/participant") | | ---------------------> | | | Ok | | | <--------------------> | ``` Implementation notes: - Initially, we thought we would be using different buckets for the local and global weights. But for now, we use the same bucket for local and global weights for now - We currently store the global weights under different keys. It turns out that this brings un-necessary complexity so we'll probably simplify this in the future - For now, the coordinator doesn't send any storage information to the participants. Thus, the participants need to be configured with the storage information. In the future, the `StartTrainingRoundResponse` could contain the endpoint url, bucket name, etc.
References:
https://xainag.atlassian.net/browse/PB-159
Needs to be merged along with:
Summary:
Remove the weights from the gRPC messages. From now on, weights will
be exchanged via s3 buckets.
The sequence diagram below illustrate this new behavior.
At the beginning of a round (1) the selected participants send a
StartTrainingRound
request, and the coordinator response with thesame
StartTrainingRoundResponse
that does not contain the globalweights anymore.
Instead, the participant fetches these weights from the store (2). S3
buckets are key-value stores, and the key for global weights is the
round number.
Then, the participant trains. Once done, it uploads its local weights
to the S3 bucket (3). The key is
<round_number>/<participant_id>
.Finally (4), the participant sends it's
EndTrainingRequest
. Beforeanswering, the coordinator retrieves the local weights the participant
has uploaded.
Important note: At the moment, the participants don't know their
ID, because the coordinator does not send it to them. Thus, they
currently generate a random ID when they start, and send it to the
coordinator so that it can retrieve the participant's weights. This is
why the
EndTrainingRoundRequest
currently has aparticipant_id
field.
At the end of the round, the coordinator writes the weights to the s3
bucket, using the next upcoming round number as key (see the sequence
diagram below).
Implementation notes:
Initially, we thought we would be using different buckets for the
local and global weights. But for now, we use the same bucket for
local and global weights for now
We currently store the global weights under different keys. It turns
out that this brings un-necessary complexity so we'll probably
simplify this in the future
For now, the coordinator doesn't send any storage information to the
participants. Thus, the participants need to be configured with the
storage information. In the future, the
StartTrainingRoundResponse
could contain the endpoint url, bucket name, etc.
Reviewer checklist
Reviewer agreement:
Merge request checklist
XP-XXX <a description in imperative form>
.XP-XXX <a description in imperative form>
.Code checklist
XP-XXX-<a_small_stub>
.