roachtest: decommission/mixed-versions failed #58058

cockroach-teamcity · 2020-12-18T07:17:40Z

(roachtest).decommission/mixed-versions failed on release-20.2@7f140b6fdc0c5086353a56016aafcdaa34ed6405:

		  | Wraps: (3) exit status 126
		  | Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *exec.ExitError: 
		  | 1: ~ ././cockroach-20.1.8 version: exit status 126
		  | (1) attached stack trace
		  |   -- stack trace:
		  |   | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.getCockroachVersion
		  |   | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:90
		  |   | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.Cockroach.Start.func1
		  |   | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:134
		  |   | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*SyncedCluster).Parallel.func1.1
		  |   | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cluster_synced.go:1651
		  |   | runtime.goexit
		  |   | 	/usr/local/go/src/runtime/asm_amd64.s:1374
		  | Wraps: (2) ~ ././cockroach-20.1.8 version
		  |   | bash: ././cockroach-20.1.8: Permission denied
		  | Wraps: (3) exit status 126
		  | Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *exec.ExitError: 
		  | 2: ~ ././cockroach-20.1.8 version: exit status 126
		  | (1) attached stack trace
		  |   -- stack trace:
		  |   | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.getCockroachVersion
		  |   | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:90
		  |   | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.Cockroach.Start.func1
		  |   | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:134
		  |   | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*SyncedCluster).Parallel.func1.1
		  |   | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cluster_synced.go:1651
		  |   | runtime.goexit
		  |   | 	/usr/local/go/src/runtime/asm_amd64.s:1374
		  | Wraps: (2) ~ ././cockroach-20.1.8 version
		  |   | bash: ././cockroach-20.1.8: Permission denied
		  | Wraps: (3) exit status 126
		  | Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *exec.ExitError: 
		  | 3: ~ ././cockroach-20.1.8 version: exit status 126
		  | (1) attached stack trace
		  |   -- stack trace:
		  |   | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.getCockroachVersion
		  |   | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:90
		  |   | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.Cockroach.Start.func1
		  |   | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:134
		  |   | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*SyncedCluster).Parallel.func1.1
		  |   | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cluster_synced.go:1651
		  |   | runtime.goexit
		  |   | 	/usr/local/go/src/runtime/asm_amd64.s:1374
		  | Wraps: (2) ~ ././cockroach-20.1.8 version
		  |   | bash: ././cockroach-20.1.8: Permission denied
		  | Wraps: (3) exit status 126
		  | Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *exec.ExitError: 
		  | I201218 07:17:25.622531 1 (gostd) cluster_synced.go:1733  command failed
		Wraps: (2) exit status 1
		Error types: (1) *main.withCommandDetails (2) *exec.ExitError

More

Artifacts: /decommission/mixed-versions

See this test on roachdash
_{powered by pkg/cmd/internal/issues}

The text was updated successfully, but these errors were encountered:

cockroach-teamcity · 2020-12-22T07:14:51Z

(roachtest).decommission/mixed-versions failed on release-20.2@cb9df011e6f1f365c03ab6a24a301fb8dcd8d708:

The test failed on branch=release-20.2, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/decommission/mixed-versions/run_1
	mixed_version_decommission.go:248,versionupgrade.go:189,mixed_version_decommission.go:121,decommission.go:73,test_runner.go:755: expected to find 1 node with membership=decommissioning, found 0
		(1) attached stack trace
		  -- stack trace:
		  | main.checkOneMembership.func1.1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/mixed_version_decommission.go:237
		  | github.com/cockroachdb/cockroach/pkg/util/retry.ForDuration
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:197
		  | main.checkOneMembership.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/mixed_version_decommission.go:228
		  | main.(*versionUpgradeTest).run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/versionupgrade.go:189
		  | main.runDecommissionMixedVersions
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/mixed_version_decommission.go:121
		  | main.registerDecommission.func3
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/decommission.go:73
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:755
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1374
		Wraps: (2) expected to find 1 node with membership=decommissioning, found 0
		Error types: (1) *withstack.withStack (2) *errutil.leafError

More

Artifacts: /decommission/mixed-versions

See this test on roachdash
_{powered by pkg/cmd/internal/issues}

cockroach-teamcity · 2021-01-05T07:13:04Z

(roachtest).decommission/mixed-versions failed on release-20.2@7274fe0d3c5ce963d2f50920670d7e6e4a2969de:

		  |   | * - Intruders can log in without password and read or write any data in the cluster.
		  |   | * - Intruders can consume all your server's resources and cause unavailability.
		  |   | *
		  |   | *
		  |   | * INFO: To start a secure server without mandating TLS for clients,
		  |   | * consider --accept-sql-without-tls instead. For other options, see:
		  |   | * 
		  |   | * - https://go.crdb.dev/issue-v/53404/v20.2
		  |   | * - https://www.cockroachlabs.com/docs/v20.2/secure-a-cluster.html
		  |   | *
		  |   | *
		  |   | * ERROR: ERROR: cockroach server exited with error: store <no-attributes>=/mnt/data1/cockroach, last used with cockroach version v19.1-1, is too old for running version v20.2 (which requires data from v20.1 or later)
		  |   | *
		  |   | ERROR: cockroach server exited with error: store <no-attributes>=/mnt/data1/cockroach, last used with cockroach version v19.1-1, is too old for running version v20.2 (which requires data from v20.1 or later)
		  |   | Failed running "start"
		  |   | E210105 07:12:47.028514 1 cli/error.go:398  ERROR: exit status 1
		  |   | ERROR: exit status 1
		  |   | Failed running "start"
		  | Wraps: (3) exit status 1
		  | Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *exec.ExitError: 
		  | I210105 07:12:47.060496 1 (gostd) cluster_synced.go:1733  command failed
		Wraps: (2) exit status 1
		Error types: (1) *main.withCommandDetails (2) *exec.ExitError

	cluster.go:1657,context.go:135,cluster.go:1646,test_runner.go:836: dead node detection: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod monitor teamcity-2558357-1609830554-13-n4cpu4 --oneshot --ignore-empty-nodes: exit status 1 2: 4403
		3: 4595
		1: 4946
		4: dead
		Error: UNCLASSIFIED_PROBLEM: 4: dead
		(1) UNCLASSIFIED_PROBLEM
		Wraps: (2) attached stack trace
		  -- stack trace:
		  | main.glob..func14
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:1143
		  | main.wrap.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:267
		  | github.com/spf13/cobra.(*Command).execute
		  | 	/home/agent/work/.go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:830
		  | github.com/spf13/cobra.(*Command).ExecuteC
		  | 	/home/agent/work/.go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:914
		  | github.com/spf13/cobra.(*Command).Execute
		  | 	/home/agent/work/.go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:864
		  | main.main
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:1839
		  | runtime.main
		  | 	/usr/local/go/src/runtime/proc.go:204
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1374
		Wraps: (3) 4: dead
		Error types: (1) errors.Unclassified (2) *withstack.withStack (3) *errutil.leafError

More

Artifacts: /decommission/mixed-versions

See this test on roachdash
_{powered by pkg/cmd/internal/issues}

cockroach-teamcity · 2021-01-27T07:19:27Z

(roachtest).decommission/mixed-versions failed on release-20.2@eb21aede5ceae7747d1392b66b61ec7799d5171c:

The test failed on branch=release-20.2, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/decommission/mixed-versions/run_1
	mixed_version_decommission.go:248,versionupgrade.go:189,mixed_version_decommission.go:121,decommission.go:73,test_runner.go:755: expected to find 1 node with membership=decommissioning, found 0
		(1) attached stack trace
		  -- stack trace:
		  | main.checkOneMembership.func1.1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/mixed_version_decommission.go:237
		  | github.com/cockroachdb/cockroach/pkg/util/retry.ForDuration
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:197
		  | main.checkOneMembership.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/mixed_version_decommission.go:228
		  | main.(*versionUpgradeTest).run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/versionupgrade.go:189
		  | main.runDecommissionMixedVersions
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/mixed_version_decommission.go:121
		  | main.registerDecommission.func3
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/decommission.go:73
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:755
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1374
		Wraps: (2) expected to find 1 node with membership=decommissioning, found 0
		Error types: (1) *withstack.withStack (2) *errutil.leafError

More

Artifacts: /decommission/mixed-versions
Related:

roachtest: decommission/mixed-versions failed #58523 roachtest: decommission/mixed-versions failed C-test-failure O-roachtest O-robot branch-master

See this test on roachdash
_{powered by pkg/cmd/internal/issues}

cockroach-teamcity · 2021-02-10T07:19:24Z

(roachtest).decommission/mixed-versions failed on release-20.2@b0012907c1bc9627ae2de83e6099c4930a32699e:

		  |   | * - Intruders can log in without password and read or write any data in the cluster.
		  |   | * - Intruders can consume all your server's resources and cause unavailability.
		  |   | *
		  |   | *
		  |   | * INFO: To start a secure server without mandating TLS for clients,
		  |   | * consider --accept-sql-without-tls instead. For other options, see:
		  |   | * 
		  |   | * - https://go.crdb.dev/issue-v/53404/v20.2
		  |   | * - https://www.cockroachlabs.com/docs/v20.2/secure-a-cluster.html
		  |   | *
		  |   | *
		  |   | * ERROR: ERROR: cockroach server exited with error: store <no-attributes>=/mnt/data1/cockroach, last used with cockroach version v19.1-1, is too old for running version v20.2 (which requires data from v20.1 or later)
		  |   | *
		  |   | ERROR: cockroach server exited with error: store <no-attributes>=/mnt/data1/cockroach, last used with cockroach version v19.1-1, is too old for running version v20.2 (which requires data from v20.1 or later)
		  |   | Failed running "start"
		  |   | E210210 07:19:07.335025 1 cli/error.go:398  ERROR: exit status 1
		  |   | ERROR: exit status 1
		  |   | Failed running "start"
		  | Wraps: (3) exit status 1
		  | Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *exec.ExitError: 
		  | I210210 07:19:07.364054 1 (gostd) cluster_synced.go:1733  command failed
		Wraps: (2) exit status 1
		Error types: (1) *main.withCommandDetails (2) *exec.ExitError

	cluster.go:1657,context.go:135,cluster.go:1646,test_runner.go:836: dead node detection: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod monitor teamcity-2661584-1612941367-04-n4cpu4 --oneshot --ignore-empty-nodes: exit status 1 2: 4755
		3: 4834
		1: 5584
		4: dead
		Error: UNCLASSIFIED_PROBLEM: 4: dead
		(1) UNCLASSIFIED_PROBLEM
		Wraps: (2) attached stack trace
		  -- stack trace:
		  | main.glob..func14
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:1143
		  | main.wrap.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:267
		  | github.com/spf13/cobra.(*Command).execute
		  | 	/home/agent/work/.go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:830
		  | github.com/spf13/cobra.(*Command).ExecuteC
		  | 	/home/agent/work/.go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:914
		  | github.com/spf13/cobra.(*Command).Execute
		  | 	/home/agent/work/.go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:864
		  | main.main
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:1839
		  | runtime.main
		  | 	/usr/local/go/src/runtime/proc.go:204
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1374
		Wraps: (3) 4: dead
		Error types: (1) errors.Unclassified (2) *withstack.withStack (3) *errutil.leafError

More

Artifacts: /decommission/mixed-versions
Related:

roachtest: decommission/mixed-versions failed #58523 roachtest: decommission/mixed-versions failed C-test-failure O-roachtest O-robot branch-master

See this test on roachdash
_{powered by pkg/cmd/internal/issues}

cockroach-teamcity · 2021-02-18T07:21:44Z

(roachtest).decommission/mixed-versions failed on release-20.2@6b701b545f1a50c8e9b2de216b0d8d02cd7d0d72:

The test failed on branch=release-20.2, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/decommission/mixed-versions/run_1
	mixed_version_decommission.go:248,versionupgrade.go:189,mixed_version_decommission.go:121,decommission.go:73,test_runner.go:755: expected to find 1 node with membership=decommissioning, found 0
		(1) attached stack trace
		  -- stack trace:
		  | main.checkOneMembership.func1.1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/mixed_version_decommission.go:237
		  | github.com/cockroachdb/cockroach/pkg/util/retry.ForDuration
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:197
		  | main.checkOneMembership.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/mixed_version_decommission.go:228
		  | main.(*versionUpgradeTest).run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/versionupgrade.go:189
		  | main.runDecommissionMixedVersions
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/mixed_version_decommission.go:121
		  | main.registerDecommission.func3
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/decommission.go:73
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:755
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1374
		Wraps: (2) expected to find 1 node with membership=decommissioning, found 0
		Error types: (1) *withstack.withStack (2) *errutil.leafError

More

Artifacts: /decommission/mixed-versions
Related:

roachtest: decommission/mixed-versions failed #58523 roachtest: decommission/mixed-versions failed C-test-failure O-roachtest O-robot branch-master

See this test on roachdash
_{powered by pkg/cmd/internal/issues}

cockroach-teamcity · 2021-03-02T07:18:23Z

(roachtest).decommission/mixed-versions failed on release-20.2@34ae57c277484ddc95e6f01137a75e0267c887b7:

The test failed on branch=release-20.2, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/decommission/mixed-versions/run_1
	mixed_version_decommission.go:248,versionupgrade.go:189,mixed_version_decommission.go:121,decommission.go:73,test_runner.go:755: expected to find 1 node with membership=decommissioning, found 0
		(1) attached stack trace
		  -- stack trace:
		  | main.checkOneMembership.func1.1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/mixed_version_decommission.go:237
		  | github.com/cockroachdb/cockroach/pkg/util/retry.ForDuration
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:197
		  | main.checkOneMembership.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/mixed_version_decommission.go:228
		  | main.(*versionUpgradeTest).run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/versionupgrade.go:189
		  | main.runDecommissionMixedVersions
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/mixed_version_decommission.go:121
		  | main.registerDecommission.func3
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/decommission.go:73
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:755
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1374
		Wraps: (2) expected to find 1 node with membership=decommissioning, found 0
		Error types: (1) *withstack.withStack (2) *errutil.leafError

More

Artifacts: /decommission/mixed-versions
Related:

roachtest: decommission/mixed-versions failed #58523 roachtest: decommission/mixed-versions failed C-test-failure O-roachtest O-robot branch-master

See this test on roachdash
_{powered by pkg/cmd/internal/issues}

knz · 2021-03-16T09:10:40Z

This is a new failure mode:

expected to find 1 node with membership=decommissioning, found 0

Probably due to recent decommissioning protocol changes.

cc @irfansharif @erikgrinaker for triage.

erikgrinaker · 2021-03-16T11:12:21Z

These errors are from before the decommissioning changes were merged in #61356. It's probably just a race where the node moves past decommissioning before the check sees it or something similar, but I'll investigate.

erikgrinaker · 2021-03-16T19:45:35Z

@tbg I suspect this (as well as #61692) is caused by or related to #56568 which wasn't backported to 20.2, as well as the follow-up self-decommissioning work in #61356. How do you feel about backporting this work? We'll hold off on the status entry removal, since that has more externally visible changes.

tbg · 2021-03-18T20:05:31Z

Self decommission only got iffy in the 21.1 cycle, this is on the 20.2 branch. Could this be something else? Wouldn't be the first time these tests need to be deflaked.

erikgrinaker · 2021-03-18T20:07:47Z

Ah ok. Will do a few local runs and see if I can reproduce.

erikgrinaker · 2021-03-19T20:22:17Z

Have been running this for a few hours, and the only failures I see are:

ERROR: cockroach server exited with error: store <no-attributes>=/mnt/data1/cockroach, last used with cockroach version v19.1-1, is too old for running version v20.2 (which requires data from v20.1 or later)

This is an oldie but a goodie, which should be fixed on master in #55240, but the 20.2 backport in #55501 was closed in favor of #55745. This apparently didn't do the trick.

As for the missing decommissioning nodes, I'll dig through the TeamCity logs for clues.

tbg · 2021-04-20T13:22:59Z

This is an oldie but a goodie, which should be fixed on master in #55240, but the 20.2 backport in #55501 was closed in favor of #55745. This apparently didn't do the trick.

From the error message, it looks as though version 20.1 is the one that didn't sync correctly. It doesn't look like we backported this all the way there.

It has issues that come from its use of the v20.1 binary, and it is unlikely that we will introduce any mixed-version issues related to decommissioning that this test would catch. Closes #58058. Release note: None

cockroach-teamcity added branch-release-20.2 C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. labels Dec 18, 2020

cockroach-teamcity added this to the 20.2 milestone Dec 18, 2020

tbg removed the release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. label Jan 4, 2021

cockroach-teamcity mentioned this issue Jan 7, 2021

roachtest: decommission/mixed-versions failed #58523

Closed

erikgrinaker self-assigned this Mar 16, 2021

tbg mentioned this issue Apr 20, 2021

release-20.2: remove decommission/mixed-versions #63922

Merged

tbg assigned tbg and unassigned erikgrinaker Apr 20, 2021

This was referenced May 1, 2021

roachtest: decommission/mixed-versions failed #64512

Closed

roachtest: decommission/mixed-versions failed #64594

Closed

tbg closed this as completed May 4, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

roachtest: decommission/mixed-versions failed #58058

roachtest: decommission/mixed-versions failed #58058

cockroach-teamcity commented Dec 18, 2020

cockroach-teamcity commented Dec 22, 2020

cockroach-teamcity commented Jan 5, 2021

cockroach-teamcity commented Jan 27, 2021

cockroach-teamcity commented Feb 10, 2021

cockroach-teamcity commented Feb 18, 2021

cockroach-teamcity commented Mar 2, 2021

knz commented Mar 16, 2021

erikgrinaker commented Mar 16, 2021

erikgrinaker commented Mar 16, 2021

tbg commented Mar 18, 2021

erikgrinaker commented Mar 18, 2021

erikgrinaker commented Mar 19, 2021

tbg commented Apr 20, 2021

roachtest: decommission/mixed-versions failed #58058

roachtest: decommission/mixed-versions failed #58058

Comments

cockroach-teamcity commented Dec 18, 2020

cockroach-teamcity commented Dec 22, 2020

cockroach-teamcity commented Jan 5, 2021

cockroach-teamcity commented Jan 27, 2021

cockroach-teamcity commented Feb 10, 2021

cockroach-teamcity commented Feb 18, 2021

cockroach-teamcity commented Mar 2, 2021

knz commented Mar 16, 2021

erikgrinaker commented Mar 16, 2021

erikgrinaker commented Mar 16, 2021

tbg commented Mar 18, 2021

erikgrinaker commented Mar 18, 2021

erikgrinaker commented Mar 19, 2021

tbg commented Apr 20, 2021