Skip to content

Commit

Permalink
storage: ensure non-expired context before each liveness update attempt
Browse files Browse the repository at this point in the history
Fixes cockroachdb#25430.

Before this change, a liveness update could get stuck in an infinite
loop if its context expired. This is because it would continue to retry
and continue to get `errRetryLiveness` errors due to
`AmbiguousResultErrors` created by `DistSender`.

Release note: None
  • Loading branch information
nvanbenschoten committed May 17, 2018
1 parent 6d0c09a commit 7d7f31d
Showing 1 changed file with 4 additions and 0 deletions.
4 changes: 4 additions & 0 deletions pkg/storage/node_liveness.go
Original file line number Diff line number Diff line change
Expand Up @@ -748,6 +748,10 @@ func (nl *NodeLiveness) updateLiveness(
handleCondFailed func(actual Liveness) error,
) error {
for {
// Before each attempt, ensure that the context has not expired.
if err := ctx.Err(); err != nil {
return err
}
for _, eng := range nl.engines {
// Synchronously writing to all disks before updating node liveness because
// we don't want any excessively slow disks to prevent the lease from
Expand Down

0 comments on commit 7d7f31d

Please sign in to comment.