Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

roachtest: acceptance/version-upgrade failed #53812

Closed
cockroach-teamcity opened this issue Sep 2, 2020 · 2 comments
Closed

roachtest: acceptance/version-upgrade failed #53812

cockroach-teamcity opened this issue Sep 2, 2020 · 2 comments
Labels
C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot.
Milestone

Comments

@cockroach-teamcity
Copy link
Member

(roachtest).acceptance/version-upgrade failed on release-19.2@ce736a5705d3a16e793cd4adc88cc87c15c97c9a:

		  |  1589.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1590.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1591.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1592.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1593.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1594.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1595.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1596.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1597.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1598.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1599.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1600.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  | _elapsed___errors__ops/sec(inst)___ops/sec(cum)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)
		  |  1601.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1602.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1603.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1604.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1605.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1606.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1607.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1608.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1609.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1610.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1611.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1612.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1613.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1614.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1615.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1616.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1617.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1618.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1619.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1620.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  | _elapsed___errors__ops/sec(inst)___ops/sec(cum)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)
		  |  1621.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1622.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1623.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1624.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1625.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1626.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1627.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1628.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1629.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		  |  1630.0s        0            0.0            0.0      0.0      0.0      0.0      0.0 opOk
		Wraps: (4) secondary error attachment
		  | signal: killed
		  | (1) signal: killed
		  | Error types: (1) *exec.ExitError
		Wraps: (5) context canceled
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *secondary.withSecondaryError (5) *errors.errorString

More

Artifacts: /acceptance/version-upgrade

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity cockroach-teamcity added branch-release-19.2 C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. labels Sep 2, 2020
@cockroach-teamcity cockroach-teamcity added this to the 20.2 milestone Sep 2, 2020
@tbg
Copy link
Member

tbg commented Sep 8, 2020

Looks like the schema change step timed out

05:54:08 mixed_version_schemachange.go:54: Workload step run: 2
05:54:08 cluster.go:2261: [w13] > ./workload run schemachange --verbose=1 --tolerate-errors=true --max-ops 10 --concurrency 2 {pgurl:1-4}
06:21:19 test_runner.go:774: [w13] dumped stacks to __stacks
06:21:19 test.go:189: test status: stopping cluster
06:21:19 cluster.go:382: > /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod stop --sig=11 teamcity-2240609-1599025771-10-n4cpu4:1-4
teamcity-2240609-1599025771-10-n4cpu4: stopping
06:21:20 test.go:337: test failure: 	test_runner.go:801: test timed out (30m0s)
06:21:20 test.go:337: [not the first failure] test failure: 
	cluster.go:2197,mixed_version_schemachange.go:67,versionupgrade.go:190,versionupgrade.go:178,acceptance.go:54,acceptance.go:91,test_runner.go:754: output in run_055408.214_n3_workload_run_schemachange: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-2240609-1599025771-10-n4cpu4:3 -- ./workload run schemachange --verbose=1 --tolerate-errors=true --max-ops 10 --concurrency 2 {pgurl:1-4} returned: context canceled
		(1) attached stack trace
		  -- stack trace:
		  | main.(*cluster).RunE

Its logs are not helpful: https://teamcity.cockroachdb.com/repository/download/Cockroach_Nightlies_WorkloadNightly/2240609:id/acceptance/version-upgrade/run_1/run_055408.214_n3_workload_run_schemachange.log

@andreimatei
Copy link
Contributor

release-19.2 and old report. closing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot.
Projects
None yet
Development

No branches or pull requests

4 participants