roachtest: tpccbench/nodes=9/cpu=4/multi-region failed

[cockroach-teamcity]

(roachtest).tpccbench/nodes=9/cpu=4/multi-region failed on release-19.1@9d9b9a1a3417ae3460b8bdc7c77bf8c9df54d899:

		          754.0s        0            9.0            6.4   2415.9   6174.0   6174.0   6174.0 delivery
		          754.0s        0           51.0           63.4   3892.3   8589.9  20401.1  27917.3 newOrder
		          754.0s        0           13.0            6.4    151.0    469.8    520.1    520.1 orderStatus
		          754.0s        0           40.0           64.1   1744.8   8589.9  11274.3  11274.3 payment
		          754.0s        0            4.0            6.3    260.0    453.0    453.0    453.0 stockLevel
		          755.0s        0            4.0            6.4   2147.5   4563.4   4563.4   4563.4 delivery
		          755.0s        0           38.0           63.4   4295.0  10737.4  28991.0  28991.0 newOrder
		          755.0s        0            5.0            6.4    671.1   1040.2   1040.2   1040.2 orderStatus
		          755.0s        0           48.0           64.1   2415.9  13958.6  22548.6  22548.6 payment
		          755.0s        0            3.0            6.3    671.1    704.6    704.6    704.6 stockLevel
		          756.0s        0            7.0            6.4   3087.0  53687.1  53687.1  53687.1 delivery
		          756.0s        0           54.0           63.4   3355.4  10737.4  11811.2  22548.6 newOrder
		          756.0s        0            7.0            6.4    209.7    503.3    503.3    503.3 orderStatus
		          756.0s        0           70.0           64.1   2013.3  18253.6  27917.3  66572.0 payment
		          756.0s        0            5.0            6.3    226.5    570.4    570.4    570.4 stockLevel
		        _elapsed___errors__ops/sec(inst)___ops/sec(cum)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)
		          757.0s        0            4.0            6.4   1879.0   2684.4   2684.4   2684.4 delivery
		          757.0s        0           10.0           63.3   4831.8   9663.7   9663.7   9663.7 newOrder
		          757.0s        0            0.0            6.4      0.0      0.0      0.0      0.0 orderStatus
		          757.0s        0           12.0           64.1   1476.4   3087.0   9663.7   9663.7 payment
		          757.0s        0            2.0            6.3     41.9    159.4    159.4    159.4 stockLevel
		          758.0s        0            1.0            6.4    973.1    973.1    973.1    973.1 delivery
		          758.0s        0            2.0           63.2   1946.2   2080.4   2080.4   2080.4 newOrder
		          758.0s        0            3.0            6.4      8.9     58.7     58.7     58.7 orderStatus
		          758.0s        0            6.0           64.0    906.0   2952.8   2952.8   2952.8 payment
		          758.0s        0            2.0            6.3    176.2    302.0    302.0    302.0 stockLevel:
		      - context canceled
		    error running tpcc load generator
		    main.runTPCCBench.func3.1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tpcc.go:818
		    main.(*monitor).Go.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2344
		    github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup.(*Group).Go.func1
		    	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57
		    runtime.goexit
		    	/usr/local/go/src/runtime/asm_amd64.s:1357

	cluster.go:1410,context.go:135,cluster.go:1399,test_runner.go:825: dead node detection: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod monitor teamcity-1816983-1584598768-82-n12cpu4-geo --oneshot --ignore-empty-nodes: exit status 1 4: skipped
		1: 18183
		2: 17637
		3: 19687
		8: skipped
		7: dead
		6: 20101
		5: 17164
		12: skipped
		11: 18087
		10: 19495
		9: 19833
		Error:  7: dead

More

Artifacts: /tpccbench/nodes=9/cpu=4/multi-region
Related:

• roachtest: tpccbench/nodes=9/cpu=4/multi-region failed #46338 roachtest: tpccbench/nodes=9/cpu=4/multi-region failed C-test-failure O-roachtest O-robot branch-provisional_202003181957_v20.1.0-beta.3 release-blocker

• roachtest: tpccbench/nodes=9/cpu=4/multi-region failed #41876 roachtest: tpccbench/nodes=9/cpu=4/multi-region failed C-test-failure O-roachtest O-robot branch-master

See this test on roachdash
_{powered by pkg/cmd/internal/issues}

[irfansharif]

Old failure.