Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cdc: node draining errors are not interpretted as retriable #46515

Closed
ajwerner opened this issue Mar 24, 2020 · 0 comments · Fixed by #49743
Closed

cdc: node draining errors are not interpretted as retriable #46515

ajwerner opened this issue Mar 24, 2020 · 0 comments · Fixed by #49743
Assignees
@ajwerner
Labels
A-cdc Change Data Capture C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.

Comments

@ajwerner
Copy link
Contributor

Describe the problem

When a CHANGEFEED fails to set up its flows due to a node draining, the CHANGEFEED will be marked as failed. CHANGEFEEDS should resume through retriable errors. Node draining should be one of those. See

cockroach/pkg/ccl/changefeedccl/errors.go

Lines 40 to 42 in c4b308d

// IsRetryableError returns true if the supplied error, or any of its parent
// causes, is a IsRetryableError.
func IsRetryableError(err error) bool {

@ajwerner ajwerner added the A-cdc Change Data Capture label Mar 24, 2020
@ajwerner ajwerner self-assigned this Mar 24, 2020
@knz knz added the C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. label May 6, 2020
@ajwerner ajwerner mentioned this issue May 28, 2020
Closed
@pbardea pbardea mentioned this issue May 31, 2020
Merged
@craig craig bot closed this as completed in a18b32f Jun 2, 2020
@miretskiy miretskiy mentioned this issue Jun 11, 2020
Merged
craig bot pushed a commit that referenced this issue Jun 11, 2020
Merge #50088
dd71687 
50088: release-20.1: cdc: Treat node draining errors as retryable. r=miretskiy a=miretskiy

Backport 1/1 commits from #49743.

/cc @cockroachdb/release

---

Fixes #46515
Fixes #43771

Handle flow registration errors due to draining node as retryable.

Release notes (reliability): Treat errors due to draining nodes
as retryable when starting CDC.


Co-authored-by: Yevgeniy Miretskiy <yevgeniy@cockroachlabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ajwerner ajwerner

Labels
A-cdc Change Data Capture C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

cdc: Treat node draining errors as retryable. miretskiy/cockroach
2 participants
@knz @ajwerner