rbd: check local image state during promote operation #2684
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
yati1998
reviewed
Nov 30, 2021
Madhu-1
force-pushed
the
dr-check-image-health
branch
from
073166d
to
0ed6411
Compare
Madhu-1
force-pushed
the
dr-check-image-health
branch
from
0ed6411
to
8bd0a7b
Compare
|
below is the output from VR condition when the image is not healthy primary conditions:
- lastTransitionTime: "2021-11-30T06:40:24Z"
message: ""
observedGeneration: 8
reason: FailedToPromote
status: "False"
type: Completed
- lastTransitionTime: "2021-11-30T06:40:24Z"
message: ""
observedGeneration: 8
reason: Error
status: "True"
type: Degraded
- lastTransitionTime: "2021-11-30T06:35:58Z"
message: ""
observedGeneration: 8
reason: NotResyncing
status: "False"
type: Resyncing
lastCompletionTime: "2021-11-30T06:35:58Z"
lastStartTime: "2021-11-26T05:50:08Z"
message: 'rpc error: code = Internal desc = replicapool/csi-vol-b2f4ee8a-4e7c-11ec-9b91-0242ac110005
is not healthy. Image state is up=false, state="unknown"' |
nixpanic
reviewed
Nov 30, 2021
Madhu-1
force-pushed
the
dr-check-image-health
branch
from
8bd0a7b
to
293ba6f
Compare
nixpanic
previously approved these changes
Nov 30, 2021
humblec
reviewed
Dec 1, 2021
Madhu-1
force-pushed
the
dr-check-image-health
branch
from
293ba6f
to
5149f30
Compare
mergify
bot
dismissed
nixpanic’s stale review
Pull request has been modified.
yati1998
reviewed
Dec 1, 2021
| func checkHealthyPrimary(ctx context.Context, rbdVol *rbdVolume) error { | ||
| mirrorStatus, err := rbdVol.getImageMirroringStatus() | ||
| if err != nil { | ||
| return err |
There was a problem hiding this comment.
return err -> return fmt.Errorf("failed to get mirror status: %w", err) ?
There was a problem hiding this comment.
getImageMirroringStatus already returning the same error message. doing this will add a duplicate error message.
Madhu-1
force-pushed
the
dr-check-image-health
branch
from
5149f30
to
39c70f7
Compare
nixpanic
approved these changes
Dec 1, 2021
humblec
reviewed
Dec 1, 2021
humblec
approved these changes
Dec 1, 2021
|
/retest all |
rbd mirroring CLI calls are async and it doesn't wait for the operation to be completed. ex:- `rbd mirror image enable` it will enable the mirroring on the image but it doesn't ensure that the image is mirroring enabled and healthy primary. The same goes for the promote volume also. This commits adds a check-in PromoteVolume to make sure the image in a healthy state i.e `up+stopped`. note:- not considering any intermediate states to make sure the image is completely healthy before responding success to the RPC call. Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Madhu-1
force-pushed
the
dr-check-image-health
branch
from
39c70f7
to
9cdd669
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
rbd mirroring CLI calls are async and it doesn't wait for the operation to be completed. ex:-
rbd mirror image enableit will enable the mirroring on the image but it doesn't ensure that the image is mirroring enabled and healthy
primary. The same goes for the promote volume also. This commits adds a check-in PromoteVolume to make sure the image is in a healthy state i.e
up+stopped.note:-
Signed-off-by: Madhu Rajanna madhupr007@gmail.com