Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jobs: TestJobControlByType failed #99200

Closed
cockroach-teamcity opened this issue Mar 22, 2023 · 51 comments
Closed

jobs: TestJobControlByType failed #99200

cockroach-teamcity opened this issue Mar 22, 2023 · 51 comments
Labels
branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-jobs
Milestone

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Mar 22, 2023

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 80c4895c566a7eaa6f16c4098980509dd3795ad7:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType2286131902
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:469: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType2286131902
--- FAIL: TestJobControlByType (69.84s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (5.67s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

/cc @cockroachdb/jobs

This test on roachdash | Improve this report!

Jira issue: CRDB-25751

@cockroach-teamcity cockroach-teamcity added branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Mar 22, 2023
@cockroach-teamcity cockroach-teamcity added this to the 23.1 milestone Mar 22, 2023
@blathers-crl blathers-crl bot added the T-jobs label Mar 22, 2023
@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 751b17200b6026b73351c57cab734415d2c4b02a:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType353813269
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:469: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType353813269
--- FAIL: TestJobControlByType (80.72s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 21
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (7.88s)
=== RUN   TestJobControlByType/pause_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/pause_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (7.73s)
=== RUN   TestJobControlByType/resume_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 3
        	            	actual  : 6
        	Test:       	TestJobControlByType/resume_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (7.25s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 751b17200b6026b73351c57cab734415d2c4b02a:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType3508735428
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:469: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType3508735428
--- FAIL: TestJobControlByType (71.67s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (5.74s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ f351747ed97862fc037717cadec23f18073fb6be:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType98672583
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:469: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType98672583
--- FAIL: TestJobControlByType (75.97s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (6.14s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ f351747ed97862fc037717cadec23f18073fb6be:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType809695496
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:469: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType809695496
--- FAIL: TestJobControlByType (80.04s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 21
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (7.63s)
=== RUN   TestJobControlByType/pause_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/pause_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (7.67s)
=== RUN   TestJobControlByType/resume_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 3
        	            	actual  : 6
        	Test:       	TestJobControlByType/resume_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (7.04s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 186218edb86b5974501b2976cfd0704d4d6d1beb:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType2871220287
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:469: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType2871220287
--- FAIL: TestJobControlByType (75.83s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 21
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (7.44s)
=== RUN   TestJobControlByType/pause_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/pause_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (6.54s)
=== RUN   TestJobControlByType/resume_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 3
        	            	actual  : 6
        	Test:       	TestJobControlByType/resume_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (6.59s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 3551dc089ac3efcb3772df2109164dcb01e79d8f:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType2298078964
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:469: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType2298078964
--- FAIL: TestJobControlByType (77.92s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (7.44s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 3551dc089ac3efcb3772df2109164dcb01e79d8f:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType2529128027
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:469: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType2529128027
--- FAIL: TestJobControlByType (69.57s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (5.85s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ a34af06d8e7230e4052db80d55da21a06013f81b:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType1260887333
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:469: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType1260887333
--- FAIL: TestJobControlByType (77.30s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (7.64s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ aec78f33d45a8376a0ecec885688bae60dbfb85c:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType1444591752
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:469: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType1444591752
--- FAIL: TestJobControlByType (72.46s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (5.81s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 7e72aae900c3ff4b44f1643c2d7ba55fbb2cbe23:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType318121920
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:469: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType318121920
--- FAIL: TestJobControlByType (79.35s)
=== RUN   TestJobControlByType/pause_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/pause_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (7.38s)
=== RUN   TestJobControlByType/resume_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 3
        	            	actual  : 6
        	Test:       	TestJobControlByType/resume_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (7.09s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 21
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (7.77s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 7e72aae900c3ff4b44f1643c2d7ba55fbb2cbe23:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType3048950648
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:469: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType3048950648
--- FAIL: TestJobControlByType (68.96s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (5.55s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 5d5abe704b0551556c35d51aa673d3162f493aa8:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType3868194235
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:470: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType3868194235
--- FAIL: TestJobControlByType (82.11s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 42
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (9.26s)
=== RUN   TestJobControlByType/pause_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/pause_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (8.51s)
=== RUN   TestJobControlByType/resume_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 3
        	            	actual  : 6
        	Test:       	TestJobControlByType/resume_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (5.47s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 5d5abe704b0551556c35d51aa673d3162f493aa8:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType2693044672
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:470: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType2693044672
--- FAIL: TestJobControlByType (81.58s)
=== RUN   TestJobControlByType/pause_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/pause_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (8.04s)
=== RUN   TestJobControlByType/resume_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 3
        	            	actual  : 6
        	Test:       	TestJobControlByType/resume_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (7.03s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 21
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (8.07s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 63b683eab3e0ebfff812bb54fdac936c101d9383:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType1083067838
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:470: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType1083067838
--- FAIL: TestJobControlByType (80.00s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (7.89s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 63b683eab3e0ebfff812bb54fdac936c101d9383:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType1972949092
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:470: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType1972949092
--- FAIL: TestJobControlByType (76.33s)
=== RUN   TestJobControlByType/pause_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/pause_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (6.69s)
=== RUN   TestJobControlByType/resume_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 3
        	            	actual  : 6
        	Test:       	TestJobControlByType/resume_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (6.76s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 21
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (7.43s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 63b683eab3e0ebfff812bb54fdac936c101d9383:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType2563412442
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:470: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType2563412442
--- FAIL: TestJobControlByType (77.93s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (7.79s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 35b724947ac464e6af45ce5193e34035fcacb1ef:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType2436905348
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:470: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType2436905348
--- FAIL: TestJobControlByType (80.62s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 21
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (7.60s)
=== RUN   TestJobControlByType/pause_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/pause_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (7.74s)
=== RUN   TestJobControlByType/resume_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 3
        	            	actual  : 6
        	Test:       	TestJobControlByType/resume_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (7.12s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 35b724947ac464e6af45ce5193e34035fcacb1ef:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType2056858158
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:470: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType2056858158
--- FAIL: TestJobControlByType (74.09s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (6.31s)
=== RUN   TestJobControlByType/resume_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 3
        	            	actual  : 6
        	Test:       	TestJobControlByType/resume_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (5.68s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ e847e4358c004d4a8977ae26749768ad1d417fba:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType3286338819
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:470: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType3286338819
--- FAIL: TestJobControlByType (78.37s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 21
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (6.51s)
=== RUN   TestJobControlByType/pause_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/pause_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (8.55s)
=== RUN   TestJobControlByType/resume_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 3
        	            	actual  : 6
        	Test:       	TestJobControlByType/resume_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (5.48s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ b432e8c20339de5cfa7c811a9ee6f5dc98d15a1e:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType4019776867
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:470: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType4019776867
--- FAIL: TestJobControlByType (75.67s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (6.08s)
=== RUN   TestJobControlByType/resume_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 3
        	            	actual  : 6
        	Test:       	TestJobControlByType/resume_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (5.65s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 571a1744c72415894dc0e742bd89e74938ea71f4:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType884145257
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:470: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType884145257
--- FAIL: TestJobControlByType (88.86s)
=== RUN   TestJobControlByType/pause_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/pause_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (8.61s)
=== RUN   TestJobControlByType/resume_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 3
        	            	actual  : 6
        	Test:       	TestJobControlByType/resume_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (7.46s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 21
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (8.06s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 571a1744c72415894dc0e742bd89e74938ea71f4:

=== RUN   TestJobControlByType
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType243824604
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestJobControlByType
    delegate_control_test.go:470: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/000f42e66f4bc5220cdea84be2328068/logTestJobControlByType243824604
--- FAIL: TestJobControlByType (84.67s)
=== RUN   TestJobControlByType/cancel_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 42
        	Test:       	TestJobControlByType/cancel_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (10.65s)
=== RUN   TestJobControlByType/pause_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 9
        	            	actual  : 18
        	Test:       	TestJobControlByType/pause_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (9.53s)
=== RUN   TestJobControlByType/resume_ALL_RESTORE_JOBS
    delegate_control_test.go:446: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/jobs/delegate_control_test.go:446
        	Error:      	Not equal: 
        	            	expected: 3
        	            	actual  : 6
        	Test:       	TestJobControlByType/resume_ALL_RESTORE_JOBS
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (5.31s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

craig bot pushed a commit that referenced this issue Apr 10, 2023
100997: jobs: add some more debugging to TestJobControlByType r=adityamaru a=knz

This test is currently flaky (#99200, #98558). However, its failures do not tell us much.

This PR adds some additional debugging.

Epic: None

Co-authored-by: Raphael 'kena' Poss <knz@thaumogen.net>
@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ a9eb6d220fc0957d4f6f8261d447991e9d03b4ad:

        	            	855648533934538753  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-11 06:13:29.81893 +0000 +0000   NULL                                    NULL                                    2023-04-11 06:13:29.81893 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:29.81893 +0000 +0000   0  NULL  NULL
        	            	855648534059220993  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-11 06:13:29.856978 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:29.856978 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:29.856978 +0000 +0000  0  NULL  NULL
        	            	855648534185148417  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-11 06:13:29.895406 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:29.895406 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:29.895406 +0000 +0000  0  NULL  NULL
        	            	855648534318907393  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-11 06:13:29.936222 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:29.936222 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:29.936222 +0000 +0000  0  NULL  NULL
        	            	855648534444736513  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-11 06:13:29.974628 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:29.974628 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:29.974628 +0000 +0000  0  NULL  NULL
        	            	855648534571646977  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-11 06:13:30.013408 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.013408 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.013408 +0000 +0000  0  NULL  NULL
        	            	855648534617686017  AUTO CREATE STATS                Table statistics refresh for system.public.role_id_seq                                  CREATE STATISTICS __auto__ FROM [48] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-11 06:13:30.027384 +0000 +0000  2023-04-11 06:13:30.128288 +0000 +0000  2023-04-11 06:13:31.945413 +0000 +0000  2023-04-11 06:13:31.945491 +0000 +0000  1  NULL    1     8048047534859406344  2023-04-11 06:13:30.128294 +0000 +0000  2023-04-12 06:13:30.128294 +0000 +0000  1  NULL  NULL
        	            	855648534701604865  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-11 06:13:30.053038 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.053038 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.053038 +0000 +0000  0  NULL  NULL
        	            	855648534840344577  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-11 06:13:30.095356 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.095356 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.095356 +0000 +0000  0  NULL  NULL
        	            	855648534978592769  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-11 06:13:30.137541 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.137541 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.137541 +0000 +0000  0  NULL  NULL
        	            	855648535117398017  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-11 06:13:30.179901 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.179901 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.179901 +0000 +0000  0  NULL  NULL
        	            	855648535290445825  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-11 06:13:30.232718 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.232718 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.232718 +0000 +0000  0  NULL  NULL
        	            	855648535430463489  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-11 06:13:30.27545 +0000 +0000   NULL                                    NULL                                    2023-04-11 06:13:30.27545 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.27545 +0000 +0000   0  NULL  NULL
        	            	855648535571070977  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-11 06:13:30.318355 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.318355 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.318355 +0000 +0000  0  NULL  NULL
        	            	855648535700406273  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-11 06:13:30.357822 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.357822 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.357822 +0000 +0000  0  NULL  NULL
        	            	855648535827218433  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-11 06:13:30.396519 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.396519 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.396519 +0000 +0000  0  NULL  NULL
        	            	855648535967727617  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-11 06:13:30.4394 +0000 +0000    NULL                                    NULL                                    2023-04-11 06:13:30.4394 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.4394 +0000 +0000    0  NULL  NULL
        	            	855648536153653249  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-11 06:13:30.496148 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.496148 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.496148 +0000 +0000  0  NULL  NULL
        	            	855648536296783873  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-11 06:13:30.539828 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.539828 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.539828 +0000 +0000  0  NULL  NULL
        	            	855648536476778497  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-11 06:13:30.594752 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.594752 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.594752 +0000 +0000  0  NULL  NULL
        	            	855648536617058305  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-11 06:13:30.637573 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.637573 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.637573 +0000 +0000  0  NULL  NULL
        	            	855648536770740225  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-11 06:13:30.68447 +0000 +0000   NULL                                    NULL                                    2023-04-11 06:13:30.68447 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.68447 +0000 +0000   0  NULL  NULL
        	            	855648536902369281  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-11 06:13:30.724641 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.724641 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.724641 +0000 +0000  0  NULL  NULL
        	            	855648537035014145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-11 06:13:30.765112 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.765112 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.765112 +0000 +0000  0  NULL  NULL
        	            	855648537162317825  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-11 06:13:30.803974 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.803974 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.803974 +0000 +0000  0  NULL  NULL
        	            	855648537294897153  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-11 06:13:30.844426 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.844426 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.844426 +0000 +0000  0  NULL  NULL
        	            	855648537416269825  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-11 06:13:30.881479 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.881479 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.881479 +0000 +0000  0  NULL  NULL
        	            	855648537541836801  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-11 06:13:30.919784 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.919784 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.919784 +0000 +0000  0  NULL  NULL
        	            	855648537669304321  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-11 06:13:30.95869 +0000 +0000   NULL                                    NULL                                    2023-04-11 06:13:30.95869 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.95869 +0000 +0000   0  NULL  NULL
        	            	855648537802276865  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-11 06:13:30.999268 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:30.999268 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:30.999268 +0000 +0000  0  NULL  NULL
        	            	855648537992101889  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-11 06:13:31.05721 +0000 +0000   NULL                                    NULL                                    2023-04-11 06:13:31.05721 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.05721 +0000 +0000   0  NULL  NULL
        	            	855648538198147073  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-11 06:13:31.12008 +0000 +0000   NULL                                    NULL                                    2023-04-11 06:13:31.12008 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.12008 +0000 +0000   0  NULL  NULL
        	            	855648538325483521  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-11 06:13:31.158928 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:31.158928 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.158928 +0000 +0000  0  NULL  NULL
        	            	855648538451116033  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-11 06:13:31.197282 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:31.197282 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.197282 +0000 +0000  0  NULL  NULL
        	            	855648538586906625  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-11 06:13:31.238723 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:31.238723 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.238723 +0000 +0000  0  NULL  NULL
        	            	855648538722402305  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-11 06:13:31.280068 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:31.280068 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.280068 +0000 +0000  0  NULL  NULL
        	            	855648538870906881  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-11 06:13:31.325386 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:31.325386 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.325386 +0000 +0000  0  NULL  NULL
        	            	855648539005648897  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-11 06:13:31.366515 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:31.366515 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.366515 +0000 +0000  0  NULL  NULL
        	            	855648539180400641  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-11 06:13:31.419849 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:31.419849 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.419849 +0000 +0000  0  NULL  NULL
        	            	855648539307966465  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-11 06:13:31.458781 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:31.458781 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.458781 +0000 +0000  0  NULL  NULL
        	            	855648539433664513  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-11 06:13:31.497134 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:31.497134 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.497134 +0000 +0000  0  NULL  NULL
        	            	855648539566047233  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-11 06:13:31.537524 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:31.537524 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.537524 +0000 +0000  0  NULL  NULL
        	            	855648539701149697  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-11 06:13:31.578755 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:31.578755 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.578755 +0000 +0000  0  NULL  NULL
        	            	855648539826454529  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-11 06:13:31.61701 +0000 +0000   NULL                                    NULL                                    2023-04-11 06:13:31.61701 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.61701 +0000 +0000   0  NULL  NULL
        	            	855648539962736641  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-11 06:13:31.658591 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:31.658591 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.658591 +0000 +0000  0  NULL  NULL
        	            	855648540093349889  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-11 06:13:31.698454 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:31.698454 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.698454 +0000 +0000  0  NULL  NULL
        	            	855648540223537153  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-11 06:13:31.738186 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:31.738186 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.738186 +0000 +0000  0  NULL  NULL
        	            	855648540349988865  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-11 06:13:31.776771 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:31.776771 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.776771 +0000 +0000  0  NULL  NULL
        	            	855648540487221249  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-11 06:13:31.818659 +0000 +0000  NULL                                    NULL                                    2023-04-11 06:13:31.818659 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-11 06:13:31.818659 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (7.33s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ c8a4703f9853a442ac676b44d074b43eb387f60c:

        	            	855984817083449345  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-12 10:43:55.291619 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:55.291619 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:55.291619 +0000 +0000  0  NULL  NULL
        	            	855984817217273857  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-12 10:43:55.332456 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:55.332456 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:55.332456 +0000 +0000  0  NULL  NULL
        	            	855984817344577537  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-12 10:43:55.371305 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:55.371305 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:55.371305 +0000 +0000  0  NULL  NULL
        	            	855984817473355777  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:43:55.410616 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:55.410616 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:55.410616 +0000 +0000  0  NULL  NULL
        	            	855984817597546497  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:43:55.44851 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:43:55.44851 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:55.44851 +0000 +0000   0  NULL  NULL
        	            	855984817723768833  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:43:55.487021 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:55.487021 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:55.487021 +0000 +0000  0  NULL  NULL
        	            	855984817858871297  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-12 10:43:55.528261 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:55.528261 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:55.528261 +0000 +0000  0  NULL  NULL
        	            	855984817985126401  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-12 10:43:55.566794 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:55.566794 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:55.566794 +0000 +0000  0  NULL  NULL
        	            	855984818111709185  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-12 10:43:55.605418 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:55.605418 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:55.605418 +0000 +0000  0  NULL  NULL
        	            	855984818238586881  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:43:55.64414 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:43:55.64414 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:55.64414 +0000 +0000   0  NULL  NULL
        	            	855984818364219393  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:43:55.682475 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:55.682475 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:55.682475 +0000 +0000  0  NULL  NULL
        	            	855984818501746689  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:43:55.724439 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:55.724439 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:55.724439 +0000 +0000  0  NULL  NULL
        	            	855984818670665729  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:43:55.776002 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:55.776002 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:55.776002 +0000 +0000  0  NULL  NULL
        	            	855984818814091265  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:43:55.819764 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:55.819764 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:55.819764 +0000 +0000  0  NULL  NULL
        	            	855984818973179905  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:43:55.868322 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:55.868322 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:55.868322 +0000 +0000  0  NULL  NULL
        	            	855984819095568385  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:43:55.90568 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:43:55.90568 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:55.90568 +0000 +0000   0  NULL  NULL
        	            	855984819218513921  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:43:55.943186 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:55.943186 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:55.943186 +0000 +0000  0  NULL  NULL
        	            	855984819342442497  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:43:55.981002 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:55.981002 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:55.981002 +0000 +0000  0  NULL  NULL
        	            	855984819467681793  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:43:56.019227 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.019227 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.019227 +0000 +0000  0  NULL  NULL
        	            	855984819588431873  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:43:56.056081 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.056081 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.056081 +0000 +0000  0  NULL  NULL
        	            	855984819748569089  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:43:56.104939 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.104939 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.104939 +0000 +0000  0  NULL  NULL
        	            	855984819870826497  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-12 10:43:56.142257 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.142257 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.142257 +0000 +0000  0  NULL  NULL
        	            	855984819997999105  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-12 10:43:56.181066 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.181066 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.181066 +0000 +0000  0  NULL  NULL
        	            	855984820120911873  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-12 10:43:56.218584 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.218584 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.218584 +0000 +0000  0  NULL  NULL
        	            	855984820275150849  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-12 10:43:56.265647 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.265647 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.265647 +0000 +0000  0  NULL  NULL
        	            	855984820395737089  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-12 10:43:56.302449 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.302449 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.302449 +0000 +0000  0  NULL  NULL
        	            	855984820546338817  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-12 10:43:56.348403 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.348403 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.348403 +0000 +0000  0  NULL  NULL
        	            	855984820675444737  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-12 10:43:56.387815 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.387815 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.387815 +0000 +0000  0  NULL  NULL
        	            	855984820807172097  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-12 10:43:56.428008 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.428008 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.428008 +0000 +0000  0  NULL  NULL
        	            	855984821098053633  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-12 10:43:56.516825 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.516825 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.516825 +0000 +0000  0  NULL  NULL
        	            	855984821266448385  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:43:56.568193 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.568193 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.568193 +0000 +0000  0  NULL  NULL
        	            	855984821424521217  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:43:56.616406 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.616406 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.616406 +0000 +0000  0  NULL  NULL
        	            	855984821570830337  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:43:56.66107 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:43:56.66107 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.66107 +0000 +0000   0  NULL  NULL
        	            	855984821721825281  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-12 10:43:56.70714 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:43:56.70714 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.70714 +0000 +0000   0  NULL  NULL
        	            	855984821877768193  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-12 10:43:56.754725 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.754725 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.754725 +0000 +0000  0  NULL  NULL
        	            	855984822013788161  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-12 10:43:56.796244 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.796244 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.796244 +0000 +0000  0  NULL  NULL
        	            	855984822154854401  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:43:56.839285 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.839285 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.839285 +0000 +0000  0  NULL  NULL
        	            	855984822290186241  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:43:56.880592 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.880592 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.880592 +0000 +0000  0  NULL  NULL
        	            	855984822434594817  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:43:56.924654 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.924654 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.924654 +0000 +0000  0  NULL  NULL
        	            	855984822563962881  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:43:56.964133 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:56.964133 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:56.964133 +0000 +0000  0  NULL  NULL
        	            	855984822692184065  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:43:57.00328 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:43:57.00328 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:57.00328 +0000 +0000   0  NULL  NULL
        	            	855984822820765697  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:43:57.04254 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:43:57.04254 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:57.04254 +0000 +0000   0  NULL  NULL
        	            	855984822953672705  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:43:57.08307 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:43:57.08307 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:57.08307 +0000 +0000   0  NULL  NULL
        	            	855984823085465601  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:43:57.123291 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:57.123291 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:57.123291 +0000 +0000  0  NULL  NULL
        	            	855984823235739649  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:43:57.169174 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:57.169174 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:57.169174 +0000 +0000  0  NULL  NULL
        	            	855984823381819393  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:43:57.213741 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:57.213741 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:57.213741 +0000 +0000  0  NULL  NULL
        	            	855984823575412737  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:43:57.272816 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:57.272816 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:57.272816 +0000 +0000  0  NULL  NULL
        	            	855984823722213377  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:43:57.317622 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:43:57.317622 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:43:57.317622 +0000 +0000  0  NULL  NULL
        	            	855984823973314561  AUTO CREATE STATS                Table statistics refresh for system.public.tenant_usage                                 CREATE STATISTICS __auto__ FROM [45] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-12 10:43:57.394209 +0000 +0000  2023-04-12 10:43:57.507189 +0000 +0000  NULL                                    2023-04-12 10:43:57.465096 +0000 +0000  0  NULL    1     822996785637398548   2023-04-12 10:43:57.507195 +0000 +0000  2023-04-13 10:43:57.507195 +0000 +0000  1  NULL  NULL
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (7.19s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ c8a4703f9853a442ac676b44d074b43eb387f60c:

        	            	855987328644382721  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-12 10:56:41.759181 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:41.759181 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:41.759181 +0000 +0000  0  NULL  NULL
        	            	855987328854097921  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:56:41.823174 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:41.823174 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:41.823174 +0000 +0000  0  NULL  NULL
        	            	855987329058242561  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:56:41.88548 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:56:41.88548 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:41.88548 +0000 +0000   0  NULL  NULL
        	            	855987329261305857  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:56:41.94745 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:56:41.94745 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:41.94745 +0000 +0000   0  NULL  NULL
        	            	855987329484587009  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-12 10:56:42.015585 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:42.015585 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:42.015585 +0000 +0000  0  NULL  NULL
        	            	855987329705934849  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-12 10:56:42.083131 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:42.083131 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:42.083131 +0000 +0000  0  NULL  NULL
        	            	855987329967259649  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-12 10:56:42.162943 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:42.162943 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:42.162943 +0000 +0000  0  NULL  NULL
        	            	855987330167570433  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:56:42.224018 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:42.224018 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:42.224018 +0000 +0000  0  NULL  NULL
        	            	855987330374172673  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:56:42.287074 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:42.287074 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:42.287074 +0000 +0000  0  NULL  NULL
        	            	855987330571239425  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:56:42.347209 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:42.347209 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:42.347209 +0000 +0000  0  NULL  NULL
        	            	855987330621538305  AUTO CREATE STATS                Table statistics refresh for system.public.rangelog                                     CREATE STATISTICS __auto__ FROM [13] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-12 10:56:42.36254 +0000 +0000   2023-04-12 10:56:42.502559 +0000 +0000  2023-04-12 10:56:43.410588 +0000 +0000  2023-04-12 10:56:43.410657 +0000 +0000  1  NULL    1     1579846869448206909  2023-04-12 10:56:42.502576 +0000 +0000  2023-04-13 10:56:42.502576 +0000 +0000  1  NULL  NULL
        	            	855987330771877889  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:56:42.408441 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:42.408441 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:42.408441 +0000 +0000  0  NULL  NULL
        	            	855987330999418881  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:56:42.477875 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:42.477875 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:42.477875 +0000 +0000  0  NULL  NULL
        	            	855987331212279809  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:56:42.542833 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:42.542833 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:42.542833 +0000 +0000  0  NULL  NULL
        	            	855987331418882049  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:56:42.605886 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:42.605886 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:42.605886 +0000 +0000  0  NULL  NULL
        	            	855987331681943553  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:56:42.686163 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:42.686163 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:42.686163 +0000 +0000  0  NULL  NULL
        	            	855987331886022657  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:56:42.748449 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:42.748449 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:42.748449 +0000 +0000  0  NULL  NULL
        	            	855987332087873537  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:56:42.810051 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:42.810051 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:42.810051 +0000 +0000  0  NULL  NULL
        	            	855987332295983105  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:56:42.87357 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:56:42.87357 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:42.87357 +0000 +0000   0  NULL  NULL
        	            	855987332505665537  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:56:42.937544 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:42.937544 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:42.937544 +0000 +0000  0  NULL  NULL
        	            	855987332736188417  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:56:43.007895 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:45.833748 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:43.007895 +0000 +0000  0  NULL  NULL
        	            	855987332981489665  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:56:43.082756 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:45.910301 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:43.082756 +0000 +0000  0  NULL  NULL
        	            	855987333202182145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:56:43.150104 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:45.986924 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:43.150104 +0000 +0000  0  NULL  NULL
        	            	855987333407277057  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:56:43.212694 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:46.358482 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:43.212694 +0000 +0000  0  NULL  NULL
        	            	855987333612077057  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:56:43.275198 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:46.433851 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:43.275198 +0000 +0000  0  NULL  NULL
        	            	855987333812092929  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:56:43.336237 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:46.511493 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:43.336237 +0000 +0000  0  NULL  NULL
        	            	855987334013681665  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-12 10:56:43.397751 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:43.397751 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:43.397751 +0000 +0000  0  NULL  NULL
        	            	855987334224805889  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-12 10:56:43.462187 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:43.462187 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:43.462187 +0000 +0000  0  NULL  NULL
        	            	855987334435635201  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-12 10:56:43.526536 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:43.526536 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:43.526536 +0000 +0000  0  NULL  NULL
        	            	855987334641745921  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:56:43.589488 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:43.589488 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:43.589488 +0000 +0000  0  NULL  NULL
        	            	855987334856998913  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:56:43.655117 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:43.655117 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:43.655117 +0000 +0000  0  NULL  NULL
        	            	855987335064485889  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:56:43.718438 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:43.718438 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:43.718438 +0000 +0000  0  NULL  NULL
        	            	855987335160037377  AUTO CREATE STATS                Table statistics refresh for system.public.span_stats_unique_keys                       CREATE STATISTICS __auto__ FROM [54] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-12 10:56:43.747586 +0000 +0000  2023-04-12 10:56:43.914537 +0000 +0000  2023-04-12 10:56:44.421975 +0000 +0000  2023-04-12 10:56:44.422026 +0000 +0000  1  NULL    1     3509196761331685520  2023-04-12 10:56:43.914554 +0000 +0000  2023-04-13 10:56:43.914554 +0000 +0000  1  NULL  NULL
        	            	855987335270039553  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:56:43.781165 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:46.112049 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:43.781165 +0000 +0000  0  NULL  NULL
        	            	855987335523139585  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:56:43.858419 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:46.196842 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:43.858419 +0000 +0000  0  NULL  NULL
        	            	855987335795998721  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:56:43.941674 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:46.274028 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:43.941674 +0000 +0000  0  NULL  NULL
        	            	855987335992344577  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:56:44.001605 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:44.001605 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:44.001605 +0000 +0000  0  NULL  NULL
        	            	855987336214904833  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:56:44.069521 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:44.069521 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:44.069521 +0000 +0000  0  NULL  NULL
        	            	855987336435040257  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:56:44.136701 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:44.136701 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:44.136701 +0000 +0000  0  NULL  NULL
        	            	855987336643575809  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:56:44.200341 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:44.200341 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:44.200341 +0000 +0000  0  NULL  NULL
        	            	855987336854405121  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:56:44.264677 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:44.264677 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:44.264677 +0000 +0000  0  NULL  NULL
        	            	855987337064185857  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:56:44.328693 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:44.328693 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:44.328693 +0000 +0000  0  NULL  NULL
        	            	855987337274654721  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:56:44.392929 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:44.392929 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:44.392929 +0000 +0000  0  NULL  NULL
        	            	855987337480994817  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:56:44.455893 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:44.455893 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:44.455893 +0000 +0000  0  NULL  NULL
        	            	855987337691234305  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:56:44.520055 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:44.520055 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:44.520055 +0000 +0000  0  NULL  NULL
        	            	855987337890398209  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:56:44.580834 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:44.580834 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:44.580834 +0000 +0000  0  NULL  NULL
        	            	855987338101751809  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:56:44.645348 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:44.645348 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:44.645348 +0000 +0000  0  NULL  NULL
        	            	855987338309894145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:56:44.708855 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:56:44.708855 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:56:44.708855 +0000 +0000  0  NULL  NULL
        	            	855987338429530113  AUTO CREATE STATS                Table statistics refresh for system.public.tenant_tasks                                 CREATE STATISTICS __auto__ FROM [59] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-12 10:56:44.745352 +0000 +0000  2023-04-12 10:56:44.872447 +0000 +0000  NULL                                    2023-04-12 10:56:44.828074 +0000 +0000  0  NULL    1     319528721392509701   2023-04-12 10:56:44.872465 +0000 +0000  2023-04-13 10:56:44.872465 +0000 +0000  1  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (9.37s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 4980b1456f185419e63c3ce43d3fda873cd7e415:

        	            	856269378818244609  AUTO CREATE STATS                Table statistics refresh for system.public.sqlliveness                                  CREATE STATISTICS __auto__ FROM [39] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-13 10:51:16.64132 +0000 +0000   2023-04-13 10:51:16.775356 +0000 +0000  2023-04-13 10:51:17.28033 +0000 +0000   2023-04-13 10:51:17.2804 +0000 +0000    1  NULL    1     7684043876381187078  2023-04-13 10:51:16.775372 +0000 +0000  2023-04-14 10:51:16.775372 +0000 +0000  1  NULL  NULL
        	            	856269378984378369  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:51:16.692039 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:16.692039 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:16.692039 +0000 +0000  0  NULL  NULL
        	            	856269379186622465  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:51:16.753758 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:16.753758 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:16.753758 +0000 +0000  0  NULL  NULL
        	            	856269379389030401  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:51:16.815525 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:16.815525 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:16.815525 +0000 +0000  0  NULL  NULL
        	            	856269379577675777  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:51:16.873092 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:16.873092 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:16.873092 +0000 +0000  0  NULL  NULL
        	            	856269379773890561  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-13 10:51:16.932978 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:16.932978 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:16.932978 +0000 +0000  0  NULL  NULL
        	            	856269379977576449  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-13 10:51:16.995148 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:16.995148 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:16.995148 +0000 +0000  0  NULL  NULL
        	            	856269380200169473  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-13 10:51:17.063066 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:17.063066 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:17.063066 +0000 +0000  0  NULL  NULL
        	            	856269380403003393  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:51:17.124971 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:17.124971 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:17.124971 +0000 +0000  0  NULL  NULL
        	            	856269380589748225  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:51:17.181952 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:17.181952 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:17.181952 +0000 +0000  0  NULL  NULL
        	            	856269380787634177  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:51:17.242344 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:17.242344 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:17.242344 +0000 +0000  0  NULL  NULL
        	            	856269380992237569  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:51:17.304788 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:17.304788 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:17.304788 +0000 +0000  0  NULL  NULL
        	            	856269381187305473  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:51:17.364316 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:17.364316 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:17.364316 +0000 +0000  0  NULL  NULL
        	            	856269381387124737  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:51:17.425288 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:17.425288 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:17.425288 +0000 +0000  0  NULL  NULL
        	            	856269381584879617  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:51:17.485647 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:17.485647 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:17.485647 +0000 +0000  0  NULL  NULL
        	            	856269381762514945  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:51:17.539857 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:17.539857 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:17.539857 +0000 +0000  0  NULL  NULL
        	            	856269381951258625  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:51:17.597454 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:17.597454 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:17.597454 +0000 +0000  0  NULL  NULL
        	            	856269382145081345  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:17.6566 +0000 +0000    NULL                                    NULL                                    2023-04-13 10:51:17.6566 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:17.6566 +0000 +0000    0  NULL  NULL
        	            	856269382339362817  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:17.715897 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:17.715897 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:17.715897 +0000 +0000  0  NULL  NULL
        	            	856269382536331265  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:17.776 +0000 +0000     NULL                                    NULL                                    2023-04-13 10:51:17.776 +0000 +0000     0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:17.776 +0000 +0000     0  NULL  NULL
        	            	856269382562545665  AUTO CREATE STATS                Table statistics refresh for system.public.span_stats_samples                           CREATE STATISTICS __auto__ FROM [56] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-13 10:51:17.78399 +0000 +0000   2023-04-13 10:51:17.9144 +0000 +0000    2023-04-13 10:51:18.391646 +0000 +0000  2023-04-13 10:51:18.391697 +0000 +0000  1  NULL    1     7538931331193170043  2023-04-13 10:51:17.914414 +0000 +0000  2023-04-14 10:51:17.914414 +0000 +0000  1  NULL  NULL
        	            	856269382737264641  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:17.837323 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:20.486393 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:17.837323 +0000 +0000  0  NULL  NULL
        	            	856269382947241985  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:17.901411 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:20.559819 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:17.901411 +0000 +0000  0  NULL  NULL
        	            	856269383148863489  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:17.962947 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:20.631257 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:17.962947 +0000 +0000  0  NULL  NULL
        	            	856269383332364289  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:18.018936 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:20.953873 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:18.018936 +0000 +0000  0  NULL  NULL
        	            	856269383556759553  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:18.087414 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:21.033874 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:18.087414 +0000 +0000  0  NULL  NULL
        	            	856269383802585089  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:18.162436 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:21.10837 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:18.162436 +0000 +0000  0  NULL  NULL
        	            	856269384028684289  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:51:18.231438 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:18.231438 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:18.231438 +0000 +0000  0  NULL  NULL
        	            	856269384228732929  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:51:18.292486 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:18.292486 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:18.292486 +0000 +0000  0  NULL  NULL
        	            	856269384421015553  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:51:18.351164 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:18.351164 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:18.351164 +0000 +0000  0  NULL  NULL
        	            	856269384608120833  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:51:18.408268 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:18.408268 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:18.408268 +0000 +0000  0  NULL  NULL
        	            	856269384829960193  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:51:18.475968 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:18.475968 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:18.475968 +0000 +0000  0  NULL  NULL
        	            	856269385020801025  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:51:18.534202 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:18.534202 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:18.534202 +0000 +0000  0  NULL  NULL
        	            	856269385221668865  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:18.595505 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:20.743458 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:18.595505 +0000 +0000  0  NULL  NULL
        	            	856269385403334657  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:18.650944 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:20.813343 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:18.650944 +0000 +0000  0  NULL  NULL
        	            	856269385590702081  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:18.708128 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:20.881052 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:18.708128 +0000 +0000  0  NULL  NULL
        	            	856269385671671809  AUTO CREATE STATS                Table statistics refresh for system.public.span_stats_tenant_boundaries                 CREATE STATISTICS __auto__ FROM [57] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-13 10:51:18.732822 +0000 +0000  2023-04-13 10:51:18.86215 +0000 +0000   2023-04-13 10:51:19.521049 +0000 +0000  2023-04-13 10:51:19.521099 +0000 +0000  1  NULL    1     583525316762502673   2023-04-13 10:51:18.862165 +0000 +0000  2023-04-14 10:51:18.862165 +0000 +0000  1  NULL  NULL
        	            	856269385786818561  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:51:18.767981 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:18.767981 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:18.767981 +0000 +0000  0  NULL  NULL
        	            	856269385996894209  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:51:18.832082 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:18.832082 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:18.832082 +0000 +0000  0  NULL  NULL
        	            	856269386190028801  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:51:18.891024 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:18.891024 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:18.891024 +0000 +0000  0  NULL  NULL
        	            	856269386392338433  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:51:18.95277 +0000 +0000   NULL                                    NULL                                    2023-04-13 10:51:18.95277 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:18.95277 +0000 +0000   0  NULL  NULL
        	            	856269386580787201  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:51:19.010285 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:19.010285 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:19.010285 +0000 +0000  0  NULL  NULL
        	            	856269386770317313  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:51:19.068116 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:19.068116 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:19.068116 +0000 +0000  0  NULL  NULL
        	            	856269386966892545  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:51:19.128102 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:19.128102 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:19.128102 +0000 +0000  0  NULL  NULL
        	            	856269387167825921  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:51:19.189431 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:19.189431 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:19.189431 +0000 +0000  0  NULL  NULL
        	            	856269387350540289  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:51:19.245187 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:19.245187 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:19.245187 +0000 +0000  0  NULL  NULL
        	            	856269387535187969  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:19.301538 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:19.301538 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:19.301538 +0000 +0000  0  NULL  NULL
        	            	856269387733401601  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:19.362022 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:19.362022 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:19.362022 +0000 +0000  0  NULL  NULL
        	            	856269387937185793  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:19.424219 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:19.424219 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:19.424219 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (8.61s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 4980b1456f185419e63c3ce43d3fda873cd7e415:

        	            	856270358210445313  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:56:15.528125 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:15.528125 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:15.528125 +0000 +0000  0  NULL  NULL
        	            	856270358342107137  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:56:15.568304 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:15.568304 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:15.568304 +0000 +0000  0  NULL  NULL
        	            	856270358480781313  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:56:15.610632 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:15.610632 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:15.610632 +0000 +0000  0  NULL  NULL
        	            	856270358607495169  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:56:15.649292 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:15.649292 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:15.649292 +0000 +0000  0  NULL  NULL
        	            	856270358733029377  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:56:15.687606 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:15.687606 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:15.687606 +0000 +0000  0  NULL  NULL
        	            	856270358855974913  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:56:15.725124 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:15.725124 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:15.725124 +0000 +0000  0  NULL  NULL
        	            	856270358977609729  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-13 10:56:15.762251 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:15.762251 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:15.762251 +0000 +0000  0  NULL  NULL
        	            	856270359097704449  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-13 10:56:15.79889 +0000 +0000   NULL                                    NULL                                    2023-04-13 10:56:15.79889 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:15.79889 +0000 +0000   0  NULL  NULL
        	            	856270359218028545  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-13 10:56:15.835626 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:15.835626 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:15.835626 +0000 +0000  0  NULL  NULL
        	            	856270359339040769  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:56:15.872551 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:15.872551 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:15.872551 +0000 +0000  0  NULL  NULL
        	            	856270359462936577  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:56:15.910348 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:15.910348 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:15.910348 +0000 +0000  0  NULL  NULL
        	            	856270359590895617  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:56:15.949404 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:15.949404 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:15.949404 +0000 +0000  0  NULL  NULL
        	            	856270359716462593  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:56:15.98772 +0000 +0000   NULL                                    NULL                                    2023-04-13 10:56:15.98772 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:15.98772 +0000 +0000   0  NULL  NULL
        	            	856270359850745857  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:56:16.028708 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:16.028708 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.028708 +0000 +0000  0  NULL  NULL
        	            	856270359985815553  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:56:16.069924 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:16.069924 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.069924 +0000 +0000  0  NULL  NULL
        	            	856270360135303169  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:56:16.115553 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:16.115553 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.115553 +0000 +0000  0  NULL  NULL
        	            	856270360342003713  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:56:16.178634 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:16.178634 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.178634 +0000 +0000  0  NULL  NULL
        	            	856270360548179969  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:56:16.241547 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:16.241547 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.241547 +0000 +0000  0  NULL  NULL
        	            	856270360695373825  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:56:16.286476 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:16.286476 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.286476 +0000 +0000  0  NULL  NULL
        	            	856270360851611649  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:56:16.334142 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:16.334142 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.334142 +0000 +0000  0  NULL  NULL
        	            	856270360975998977  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:56:16.372108 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:16.372108 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.372108 +0000 +0000  0  NULL  NULL
        	            	856270361101074433  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:56:16.410283 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:20.920813 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.410283 +0000 +0000  0  NULL  NULL
        	            	856270361316917249  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:56:16.476152 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:20.99103 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.476152 +0000 +0000  0  NULL  NULL
        	            	856270361450479617  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:56:16.516905 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:21.058495 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.516905 +0000 +0000  0  NULL  NULL
        	            	856270361575915521  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:56:16.555194 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:21.897295 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.555194 +0000 +0000  0  NULL  NULL
        	            	856270361700401153  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:56:16.593189 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:21.971676 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.593189 +0000 +0000  0  NULL  NULL
        	            	856270361819873281  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:56:16.629637 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:22.04038 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.629637 +0000 +0000  0  NULL  NULL
        	            	856270361944260609  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:56:16.6676 +0000 +0000    NULL                                    NULL                                    2023-04-13 10:56:16.6676 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.6676 +0000 +0000    0  NULL  NULL
        	            	856270362076086273  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:56:16.707834 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:16.707834 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.707834 +0000 +0000  0  NULL  NULL
        	            	856270362204405761  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:56:16.746982 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:16.746982 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.746982 +0000 +0000  0  NULL  NULL
        	            	856270362327154689  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:56:16.784466 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:16.784466 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.784466 +0000 +0000  0  NULL  NULL
        	            	856270362460323841  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:56:16.825093 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:16.825093 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.825093 +0000 +0000  0  NULL  NULL
        	            	856270362592378881  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:56:16.865393 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:16.865393 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.865393 +0000 +0000  0  NULL  NULL
        	            	856270362712670209  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:56:16.902096 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:21.599571 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.902096 +0000 +0000  0  NULL  NULL
        	            	856270362837483521  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:56:16.940181 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:21.695508 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.940181 +0000 +0000  0  NULL  NULL
        	            	856270362958561281  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:56:16.977136 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:21.801704 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:16.977136 +0000 +0000  0  NULL  NULL
        	            	856270363080425473  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:56:17.014334 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:17.014334 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:17.014334 +0000 +0000  0  NULL  NULL
        	            	856270363203371009  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:56:17.051849 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:17.051849 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:17.051849 +0000 +0000  0  NULL  NULL
        	            	856270363345518593  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:56:17.09524 +0000 +0000   NULL                                    NULL                                    2023-04-13 10:56:17.09524 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:17.09524 +0000 +0000   0  NULL  NULL
        	            	856270363494973441  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:56:17.140845 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:17.140845 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:17.140845 +0000 +0000  0  NULL  NULL
        	            	856270363633811457  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:56:17.183207 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:17.183207 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:17.183207 +0000 +0000  0  NULL  NULL
        	            	856270363754692609  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:56:17.220099 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:17.220099 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:17.220099 +0000 +0000  0  NULL  NULL
        	            	856270363887632385  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:56:17.260674 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:17.260674 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:17.260674 +0000 +0000  0  NULL  NULL
        	            	856270364012150785  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:56:17.298667 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:17.298667 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:17.298667 +0000 +0000  0  NULL  NULL
        	            	856270364140109825  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:56:17.337722 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:17.337722 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:17.337722 +0000 +0000  0  NULL  NULL
        	            	856270364272164865  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:56:17.378029 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:17.378029 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:17.378029 +0000 +0000  0  NULL  NULL
        	            	856270364401139713  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:56:17.417378 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:17.417378 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:17.417378 +0000 +0000  0  NULL  NULL
        	            	856270364533325825  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:56:17.457713 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:56:17.457713 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:56:17.457713 +0000 +0000  0  NULL  NULL
        	            	856270365335748609  AUTO CREATE STATS                Table statistics refresh for system.public.statement_activity                           CREATE STATISTICS __auto__ FROM [60] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-13 10:56:17.702594 +0000 +0000  2023-04-13 10:56:17.815911 +0000 +0000  NULL                                    2023-04-13 10:56:17.779699 +0000 +0000  0  NULL    1     6756487832397359705  2023-04-13 10:56:17.815917 +0000 +0000  2023-04-14 10:56:17.815917 +0000 +0000  1  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (10.16s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ ad16885ca3b4567ed5eb34646fe8281fd2d740e3:

        	            	856549949296508929  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-14 10:38:19.956235 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:19.956235 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:19.956235 +0000 +0000  0  NULL  NULL
        	            	856549949548625921  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-14 10:38:20.033225 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:20.033225 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:20.033225 +0000 +0000  0  NULL  NULL
        	            	856549949780000769  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-14 10:38:20.103796 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:20.103796 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:20.103796 +0000 +0000  0  NULL  NULL
        	            	856549949875453953  AUTO CREATE STATS                Table statistics refresh for system.public.transaction_statistics                       CREATE STATISTICS __auto__ FROM [43] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-14 10:38:20.132898 +0000 +0000  2023-04-14 10:38:20.267146 +0000 +0000  2023-04-14 10:38:21.51827 +0000 +0000   2023-04-14 10:38:21.518343 +0000 +0000  1  NULL    1     7479319477333447422  2023-04-14 10:38:20.26716 +0000 +0000   2023-04-15 10:38:20.26716 +0000 +0000   1  NULL  NULL
        	            	856549949989224449  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-14 10:38:20.167632 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:20.167632 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:20.167632 +0000 +0000  0  NULL  NULL
        	            	856549950189109249  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-14 10:38:20.228637 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:20.228637 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:20.228637 +0000 +0000  0  NULL  NULL
        	            	856549950413996033  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-14 10:38:20.297267 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:20.297267 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:20.297267 +0000 +0000  0  NULL  NULL
        	            	856549950612930561  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-14 10:38:20.357976 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:20.357976 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:20.357976 +0000 +0000  0  NULL  NULL
        	            	856549950809014273  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-14 10:38:20.417814 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:20.417814 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:20.417814 +0000 +0000  0  NULL  NULL
        	            	856549951003820033  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-14 10:38:20.477265 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:20.477265 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:20.477265 +0000 +0000  0  NULL  NULL
        	            	856549951206129665  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-14 10:38:20.539011 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:20.539011 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:20.539011 +0000 +0000  0  NULL  NULL
        	            	856549951404638209  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-14 10:38:20.599582 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:20.599582 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:20.599582 +0000 +0000  0  NULL  NULL
        	            	856549951603769345  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-14 10:38:20.660354 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:20.660354 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:20.660354 +0000 +0000  0  NULL  NULL
        	            	856549951813353473  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-14 10:38:20.724314 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:20.724314 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:20.724314 +0000 +0000  0  NULL  NULL
        	            	856549952026345473  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-14 10:38:20.789318 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:20.789318 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:20.789318 +0000 +0000  0  NULL  NULL
        	            	856549952255885313  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-14 10:38:20.859367 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:20.859367 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:20.859367 +0000 +0000  0  NULL  NULL
        	            	856549952502136833  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-14 10:38:20.934514 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:20.934514 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:20.934514 +0000 +0000  0  NULL  NULL
        	            	856549952701038593  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-14 10:38:20.995212 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:20.995212 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:20.995212 +0000 +0000  0  NULL  NULL
        	            	856549952907345921  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 10:38:21.058177 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:21.058177 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:21.058177 +0000 +0000  0  NULL  NULL
        	            	856549953111293953  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 10:38:21.12042 +0000 +0000   NULL                                    NULL                                    2023-04-14 10:38:21.12042 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:21.12042 +0000 +0000   0  NULL  NULL
        	            	856549953296171009  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 10:38:21.176836 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:21.176836 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:21.176836 +0000 +0000  0  NULL  NULL
        	            	856549953498841089  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-14 10:38:21.23869 +0000 +0000   NULL                                    NULL                                    2023-04-14 10:38:21.23869 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:21.23869 +0000 +0000   0  NULL  NULL
        	            	856549953701150721  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-14 10:38:21.300429 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:21.300429 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:21.300429 +0000 +0000  0  NULL  NULL
        	            	856549953904017409  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-14 10:38:21.362343 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:21.362343 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:21.362343 +0000 +0000  0  NULL  NULL
        	            	856549954105212929  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-14 10:38:21.423737 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:21.423737 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:21.423737 +0000 +0000  0  NULL  NULL
        	            	856549954307817473  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-14 10:38:21.48557 +0000 +0000   NULL                                    NULL                                    2023-04-14 10:38:21.48557 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:21.48557 +0000 +0000   0  NULL  NULL
        	            	856549954506358785  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-14 10:38:21.546159 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:21.546159 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:21.546159 +0000 +0000  0  NULL  NULL
        	            	856549954713288705  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-14 10:38:21.609309 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:21.609309 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:21.609309 +0000 +0000  0  NULL  NULL
        	            	856549954912649217  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-14 10:38:21.670148 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:21.670148 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:21.670148 +0000 +0000  0  NULL  NULL
        	            	856549955119677441  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-14 10:38:21.733335 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:21.733335 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:21.733335 +0000 +0000  0  NULL  NULL
        	            	856549955337388033  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-14 10:38:21.799765 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:21.799765 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:21.799765 +0000 +0000  0  NULL  NULL
        	            	856549955538321409  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-14 10:38:21.861086 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:21.861086 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:21.861086 +0000 +0000  0  NULL  NULL
        	            	856549955736141825  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-14 10:38:21.921455 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:21.921455 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:21.921455 +0000 +0000  0  NULL  NULL
        	            	856549955995631617  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-14 10:38:22.000651 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:22.000651 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:22.000651 +0000 +0000  0  NULL  NULL
        	            	856549956224155649  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-14 10:38:22.070394 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:22.070394 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:22.070394 +0000 +0000  0  NULL  NULL
        	            	856549956371578881  AUTO CREATE STATS                Table statistics refresh for system.public.sql_instances                                CREATE STATISTICS __auto__ FROM [46] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-14 10:38:22.115362 +0000 +0000  2023-04-14 10:38:22.255641 +0000 +0000  2023-04-14 10:38:22.959925 +0000 +0000  2023-04-14 10:38:22.959981 +0000 +0000  1  NULL    1     2634426780498093303  2023-04-14 10:38:22.255657 +0000 +0000  2023-04-15 10:38:22.255657 +0000 +0000  1  NULL  NULL
        	            	856549956457562113  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-14 10:38:22.141619 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:22.141619 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:22.141619 +0000 +0000  0  NULL  NULL
        	            	856549956663705601  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-14 10:38:22.204528 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:22.204528 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:22.204528 +0000 +0000  0  NULL  NULL
        	            	856549956869718017  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-14 10:38:22.267396 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:22.267396 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:22.267396 +0000 +0000  0  NULL  NULL
        	            	856549957074092033  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-14 10:38:22.329776 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:22.329776 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:22.329776 +0000 +0000  0  NULL  NULL
        	            	856549957275189249  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-14 10:38:22.391149 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:22.391149 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:22.391149 +0000 +0000  0  NULL  NULL
        	            	856549957470355457  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-14 10:38:22.45069 +0000 +0000   NULL                                    NULL                                    2023-04-14 10:38:22.45069 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:22.45069 +0000 +0000   0  NULL  NULL
        	            	856549957693177857  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-14 10:38:22.518707 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:22.518707 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:22.518707 +0000 +0000  0  NULL  NULL
        	            	856549957895553025  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-14 10:38:22.580452 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:22.580452 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:22.580452 +0000 +0000  0  NULL  NULL
        	            	856549958098649089  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-14 10:38:22.642441 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:22.642441 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:22.642441 +0000 +0000  0  NULL  NULL
        	            	856549958303383553  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-14 10:38:22.704924 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:22.704924 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:22.704924 +0000 +0000  0  NULL  NULL
        	            	856549958518636545  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 10:38:22.770629 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:22.770629 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:22.770629 +0000 +0000  0  NULL  NULL
        	            	856549958723764225  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 10:38:22.833205 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:22.833205 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:22.833205 +0000 +0000  0  NULL  NULL
        	            	856549958946521089  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 10:38:22.901191 +0000 +0000  NULL                                    NULL                                    2023-04-14 10:38:22.901191 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 10:38:22.901191 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (7.79s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 84905b8396e7cd7ef999b264faaafe13110d631a:

        	            	856829538131935233  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-15 10:20:23.697513 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:23.697513 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:23.697513 +0000 +0000  0  NULL  NULL
        	            	856829538327199745  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-15 10:20:23.757104 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:23.757104 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:23.757104 +0000 +0000  0  NULL  NULL
        	            	856829538542452737  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-15 10:20:23.822794 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:23.822794 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:23.822794 +0000 +0000  0  NULL  NULL
        	            	856829538776809473  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-15 10:20:23.89432 +0000 +0000   NULL                                    NULL                                    2023-04-15 10:20:23.89432 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:23.89432 +0000 +0000   0  NULL  NULL
        	            	856829538991734785  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-15 10:20:23.959899 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:23.959899 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:23.959899 +0000 +0000  0  NULL  NULL
        	            	856829539217080321  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-15 10:20:24.028671 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:24.028671 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:24.028671 +0000 +0000  0  NULL  NULL
        	            	856829539445964801  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-15 10:20:24.098522 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:24.098522 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:24.098522 +0000 +0000  0  NULL  NULL
        	            	856829539693723649  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-15 10:20:24.174134 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:24.174134 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:24.174134 +0000 +0000  0  NULL  NULL
        	            	856829539916972033  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-15 10:20:24.242257 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:24.242257 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:24.242257 +0000 +0000  0  NULL  NULL
        	            	856829540132618241  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-15 10:20:24.308069 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:24.308069 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:24.308069 +0000 +0000  0  NULL  NULL
        	            	856829540338630657  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-15 10:20:24.370937 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:24.370937 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:24.370937 +0000 +0000  0  NULL  NULL
        	            	856829540560601089  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-15 10:20:24.438676 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:24.438676 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:24.438676 +0000 +0000  0  NULL  NULL
        	            	856829540778180609  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-15 10:20:24.505135 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:24.505135 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:24.505135 +0000 +0000  0  NULL  NULL
        	            	856829540993794049  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-15 10:20:24.570879 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:24.570879 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:24.570879 +0000 +0000  0  NULL  NULL
        	            	856829541202493441  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-15 10:20:24.634575 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:24.634575 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:24.634575 +0000 +0000  0  NULL  NULL
        	            	856829541423448065  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-15 10:20:24.702 +0000 +0000     NULL                                    NULL                                    2023-04-15 10:20:24.702 +0000 +0000     0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:24.702 +0000 +0000     0  NULL  NULL
        	            	856829541677891585  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-15 10:20:24.779684 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:24.779684 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:24.779684 +0000 +0000  0  NULL  NULL
        	            	856829541917327361  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:20:24.852714 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:24.852714 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:24.852714 +0000 +0000  0  NULL  NULL
        	            	856829542045024257  AUTO CREATE STATS                Table statistics refresh for system.public.statement_activity                           CREATE STATISTICS __auto__ FROM [60] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-15 10:20:24.891685 +0000 +0000  2023-04-15 10:20:25.055002 +0000 +0000  2023-04-15 10:20:26.528469 +0000 +0000  2023-04-15 10:20:26.528542 +0000 +0000  1  NULL    1     8061135960973982525  2023-04-15 10:20:25.055011 +0000 +0000  2023-04-16 10:20:25.055011 +0000 +0000  1  NULL  NULL
        	            	856829542156042241  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:20:24.925579 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:24.925579 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:24.925579 +0000 +0000  0  NULL  NULL
        	            	856829542397444097  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:20:24.999246 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:24.999246 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:24.999246 +0000 +0000  0  NULL  NULL
        	            	856829542631440385  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:20:25.070666 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:27.941801 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:25.070666 +0000 +0000  0  NULL  NULL
        	            	856829542847414273  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:20:25.136566 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:28.017063 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:25.136566 +0000 +0000  0  NULL  NULL
        	            	856829543057555457  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:20:25.200699 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:28.093546 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:25.200699 +0000 +0000  0  NULL  NULL
        	            	856829543270252545  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:20:25.265604 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:28.436182 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:25.265604 +0000 +0000  0  NULL  NULL
        	            	856829543478624257  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:20:25.329201 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:28.50923 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:25.329201 +0000 +0000  0  NULL  NULL
        	            	856829543697448961  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:20:25.39598 +0000 +0000   NULL                                    NULL                                    2023-04-15 10:20:28.581746 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:25.39598 +0000 +0000   0  NULL  NULL
        	            	856829543924858881  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-15 10:20:25.465373 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:25.465373 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:25.465373 +0000 +0000  0  NULL  NULL
        	            	856829544210595841  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-15 10:20:25.552579 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:25.552579 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:25.552579 +0000 +0000  0  NULL  NULL
        	            	856829544420212737  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-15 10:20:25.616542 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:25.616542 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:25.616542 +0000 +0000  0  NULL  NULL
        	            	856829544634023937  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-15 10:20:25.6818 +0000 +0000    NULL                                    NULL                                    2023-04-15 10:20:25.6818 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:25.6818 +0000 +0000    0  NULL  NULL
        	            	856829544841150465  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-15 10:20:25.744998 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:25.744998 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:25.744998 +0000 +0000  0  NULL  NULL
        	            	856829545051127809  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-15 10:20:25.809089 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:25.809089 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:25.809089 +0000 +0000  0  NULL  NULL
        	            	856829545258123265  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:20:25.872255 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:28.204939 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:25.872255 +0000 +0000  0  NULL  NULL
        	            	856829545471836161  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:20:25.937479 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:28.28054 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:25.937479 +0000 +0000  0  NULL  NULL
        	            	856829545702752257  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:20:26.007947 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:28.354805 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:26.007947 +0000 +0000  0  NULL  NULL
        	            	856829545911091201  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-15 10:20:26.071533 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:26.071533 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:26.071533 +0000 +0000  0  NULL  NULL
        	            	856829546123132929  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-15 10:20:26.136232 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:26.136232 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:26.136232 +0000 +0000  0  NULL  NULL
        	            	856829546339893249  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-15 10:20:26.202399 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:26.202399 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:26.202399 +0000 +0000  0  NULL  NULL
        	            	856829546546429953  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-15 10:20:26.265414 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:26.265414 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:26.265414 +0000 +0000  0  NULL  NULL
        	            	856829546758307841  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-15 10:20:26.330069 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:26.330069 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:26.330069 +0000 +0000  0  NULL  NULL
        	            	856829547081957377  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-15 10:20:26.428844 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:26.428844 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:26.428844 +0000 +0000  0  NULL  NULL
        	            	856829547323392001  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-15 10:20:26.502526 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:26.502526 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:26.502526 +0000 +0000  0  NULL  NULL
        	            	856829547535990785  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-15 10:20:26.567403 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:26.567403 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:26.567403 +0000 +0000  0  NULL  NULL
        	            	856829547750916097  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-15 10:20:26.632995 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:26.632995 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:26.632995 +0000 +0000  0  NULL  NULL
        	            	856829547960205313  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:20:26.696871 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:26.696871 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:26.696871 +0000 +0000  0  NULL  NULL
        	            	856829548176769025  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:20:26.762959 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:26.762959 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:26.762959 +0000 +0000  0  NULL  NULL
        	            	856829548406374401  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:20:26.833028 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:20:26.833028 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:20:26.833028 +0000 +0000  0  NULL  NULL
        	            	856829548443598849  AUTO CREATE STATS                Table statistics refresh for system.public.eventlog                                     CREATE STATISTICS __auto__ FROM [12] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-15 10:20:26.844376 +0000 +0000  2023-04-15 10:20:26.986451 +0000 +0000  NULL                                    2023-04-15 10:20:26.934472 +0000 +0000  0  NULL    1     6633303886227743900  2023-04-15 10:20:26.986466 +0000 +0000  2023-04-16 10:20:26.986466 +0000 +0000  1  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (9.37s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 84905b8396e7cd7ef999b264faaafe13110d631a:

        	            	856832787791773697  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-15 10:36:55.415011 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:55.415011 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:55.415011 +0000 +0000  0  NULL  NULL
        	            	856832787930382337  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-15 10:36:55.457308 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:55.457308 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:55.457308 +0000 +0000  0  NULL  NULL
        	            	856832788072759297  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-15 10:36:55.50076 +0000 +0000   NULL                                    NULL                                    2023-04-15 10:36:55.50076 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:55.50076 +0000 +0000   0  NULL  NULL
        	            	856832788214546433  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-15 10:36:55.544057 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:55.544057 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:55.544057 +0000 +0000  0  NULL  NULL
        	            	856832788364460033  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-15 10:36:55.589782 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:55.589782 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:55.589782 +0000 +0000  0  NULL  NULL
        	            	856832788507262977  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-15 10:36:55.633358 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:55.633358 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:55.633358 +0000 +0000  0  NULL  NULL
        	            	856832788640989185  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-15 10:36:55.674166 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:55.674166 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:55.674166 +0000 +0000  0  NULL  NULL
        	            	856832788783104001  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-15 10:36:55.717533 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:55.717533 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:55.717533 +0000 +0000  0  NULL  NULL
        	            	856832788928856065  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-15 10:36:55.762015 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:55.762015 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:55.762015 +0000 +0000  0  NULL  NULL
        	            	856832789070315521  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-15 10:36:55.805182 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:55.805182 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:55.805182 +0000 +0000  0  NULL  NULL
        	            	856832789212364801  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-15 10:36:55.84854 +0000 +0000   NULL                                    NULL                                    2023-04-15 10:36:55.84854 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:55.84854 +0000 +0000   0  NULL  NULL
        	            	856832789346811905  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-15 10:36:55.889559 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:55.889559 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:55.889559 +0000 +0000  0  NULL  NULL
        	            	856832789479456769  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-15 10:36:55.930048 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:55.930048 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:55.930048 +0000 +0000  0  NULL  NULL
        	            	856832789613936641  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-15 10:36:55.971086 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:55.971086 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:55.971086 +0000 +0000  0  NULL  NULL
        	            	856832789763063809  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-15 10:36:56.016605 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.016605 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.016605 +0000 +0000  0  NULL  NULL
        	            	856832789921857537  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-15 10:36:56.06506 +0000 +0000   NULL                                    NULL                                    2023-04-15 10:36:56.06506 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.06506 +0000 +0000   0  NULL  NULL
        	            	856832790075408385  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-15 10:36:56.111926 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.111926 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.111926 +0000 +0000  0  NULL  NULL
        	            	856832790282502145  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-15 10:36:56.175116 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.175116 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.175116 +0000 +0000  0  NULL  NULL
        	            	856832790421078017  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:36:56.217432 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.217432 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.217432 +0000 +0000  0  NULL  NULL
        	            	856832790603399169  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:36:56.273048 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.273048 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.273048 +0000 +0000  0  NULL  NULL
        	            	856832790736109569  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:36:56.313548 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.313548 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.313548 +0000 +0000  0  NULL  NULL
        	            	856832790872293377  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-15 10:36:56.355111 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.355111 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.355111 +0000 +0000  0  NULL  NULL
        	            	856832791051665409  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-15 10:36:56.40987 +0000 +0000   NULL                                    NULL                                    2023-04-15 10:36:56.40987 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.40987 +0000 +0000   0  NULL  NULL
        	            	856832791212163073  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-15 10:36:56.458831 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.458831 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.458831 +0000 +0000  0  NULL  NULL
        	            	856832791358537729  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-15 10:36:56.503491 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.503491 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.503491 +0000 +0000  0  NULL  NULL
        	            	856832791489609729  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-15 10:36:56.543493 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.543493 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.543493 +0000 +0000  0  NULL  NULL
        	            	856832791623041025  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-15 10:36:56.584242 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.584242 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.584242 +0000 +0000  0  NULL  NULL
        	            	856832791754276865  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-15 10:36:56.624267 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.624267 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.624267 +0000 +0000  0  NULL  NULL
        	            	856832791893704705  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-15 10:36:56.666821 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.666821 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.666821 +0000 +0000  0  NULL  NULL
        	            	856832792035393537  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-15 10:36:56.710067 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.710067 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.710067 +0000 +0000  0  NULL  NULL
        	            	856832792174493697  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-15 10:36:56.752509 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.752509 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.752509 +0000 +0000  0  NULL  NULL
        	            	856832792311988225  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-15 10:36:56.79447 +0000 +0000   NULL                                    NULL                                    2023-04-15 10:36:56.79447 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.79447 +0000 +0000   0  NULL  NULL
        	            	856832792448663553  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-15 10:36:56.836181 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.836181 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.836181 +0000 +0000  0  NULL  NULL
        	            	856832792643076097  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-15 10:36:56.895511 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.895511 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.895511 +0000 +0000  0  NULL  NULL
        	            	856832792781160449  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-15 10:36:56.937654 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.937654 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.937654 +0000 +0000  0  NULL  NULL
        	            	856832792956207105  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-15 10:36:56.991078 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:56.991078 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:56.991078 +0000 +0000  0  NULL  NULL
        	            	856832793112576001  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-15 10:36:57.03879 +0000 +0000   NULL                                    NULL                                    2023-04-15 10:36:57.03879 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:57.03879 +0000 +0000   0  NULL  NULL
        	            	856832793250889729  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-15 10:36:57.081011 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:57.081011 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:57.081011 +0000 +0000  0  NULL  NULL
        	            	856832793386614785  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-15 10:36:57.122423 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:57.122423 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:57.122423 +0000 +0000  0  NULL  NULL
        	            	856832793521029121  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-15 10:36:57.163439 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:57.163439 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:57.163439 +0000 +0000  0  NULL  NULL
        	            	856832793668354049  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-15 10:36:57.208414 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:57.208414 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:57.208414 +0000 +0000  0  NULL  NULL
        	            	856832793802014721  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-15 10:36:57.249185 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:57.249185 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:57.249185 +0000 +0000  0  NULL  NULL
        	            	856832793939247105  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-15 10:36:57.291067 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:57.291067 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:57.291067 +0000 +0000  0  NULL  NULL
        	            	856832794074972161  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-15 10:36:57.332496 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:57.332496 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:57.332496 +0000 +0000  0  NULL  NULL
        	            	856832794208174081  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-15 10:36:57.373141 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:57.373141 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:57.373141 +0000 +0000  0  NULL  NULL
        	            	856832794347372545  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:36:57.415637 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:57.415637 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:57.415637 +0000 +0000  0  NULL  NULL
        	            	856832794487980033  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:36:57.458535 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:57.458535 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:57.458535 +0000 +0000  0  NULL  NULL
        	            	856832794628128769  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:36:57.501302 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:36:57.501302 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:36:57.501302 +0000 +0000  0  NULL  NULL
        	            	856832795150450689  AUTO CREATE STATS                Table statistics refresh for system.public.statement_activity                           CREATE STATISTICS __auto__ FROM [60] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-15 10:36:57.660668 +0000 +0000  2023-04-15 10:36:57.778839 +0000 +0000  NULL                                    2023-04-15 10:36:57.74034 +0000 +0000   0  NULL    1     8474370518126023761  2023-04-15 10:36:57.778845 +0000 +0000  2023-04-16 10:36:57.778845 +0000 +0000  1  NULL  NULL
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (8.08s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 6cce6c746150307a9ecf3b529dfd633a6985c110:

        	            	857106497621458945  AUTO CREATE STATS                Table statistics refresh for system.public.sqlliveness                                  CREATE STATISTICS __auto__ FROM [39] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-16 09:49:05.026091 +0000 +0000  2023-04-16 09:49:05.172396 +0000 +0000  2023-04-16 09:49:05.804503 +0000 +0000  2023-04-16 09:49:05.804553 +0000 +0000  1  NULL    1     3830654253038817505  2023-04-16 09:49:05.172414 +0000 +0000  2023-04-17 09:49:05.172414 +0000 +0000  1  NULL  NULL
        	            	857106497633320961  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:49:05.029742 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:05.029742 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:05.029742 +0000 +0000  0  NULL  NULL
        	            	857106497865383937  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:49:05.100545 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:05.100545 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:05.100545 +0000 +0000  0  NULL  NULL
        	            	857106498098331649  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:49:05.171647 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:05.171647 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:05.171647 +0000 +0000  0  NULL  NULL
        	            	857106498332426241  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:49:05.243079 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:05.243079 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:05.243079 +0000 +0000  0  NULL  NULL
        	            	857106498568814593  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:49:05.315221 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:05.315221 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:05.315221 +0000 +0000  0  NULL  NULL
        	            	857106498841772033  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:49:05.398525 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:05.398525 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:05.398525 +0000 +0000  0  NULL  NULL
        	            	857106499085729793  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:49:05.472965 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:05.472965 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:05.472965 +0000 +0000  0  NULL  NULL
        	            	857106499360587777  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:49:05.556876 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:05.556876 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:05.556876 +0000 +0000  0  NULL  NULL
        	            	857106499603529729  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:49:05.630989 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:05.630989 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:05.630989 +0000 +0000  0  NULL  NULL
        	            	857106499850338305  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:49:05.706302 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:05.706302 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:05.706302 +0000 +0000  0  NULL  NULL
        	            	857106500093411329  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:49:05.780483 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:05.780483 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:05.780483 +0000 +0000  0  NULL  NULL
        	            	857106500328095745  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:49:05.852101 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:05.852101 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:05.852101 +0000 +0000  0  NULL  NULL
        	            	857106500565008385  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:49:05.924407 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:05.924407 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:05.924407 +0000 +0000  0  NULL  NULL
        	            	857106500802347009  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:49:05.99683 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:49:05.99683 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:05.99683 +0000 +0000   0  NULL  NULL
        	            	857106501036703745  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:49:06.068355 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:06.068355 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:06.068355 +0000 +0000  0  NULL  NULL
        	            	857106501281611777  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:49:06.143093 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:06.143093 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:06.143093 +0000 +0000  0  NULL  NULL
        	            	857106501291540481  AUTO CREATE STATS                Table statistics refresh for system.public.span_stats_samples                           CREATE STATISTICS __auto__ FROM [56] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-16 09:49:06.146114 +0000 +0000  2023-04-16 09:49:06.296381 +0000 +0000  2023-04-16 09:49:06.842612 +0000 +0000  2023-04-16 09:49:06.842739 +0000 +0000  1  NULL    1     6412038739091102512  2023-04-16 09:49:06.296395 +0000 +0000  2023-04-17 09:49:06.296395 +0000 +0000  1  NULL  NULL
        	            	857106501516853249  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:49:06.214886 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:06.214886 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:06.214886 +0000 +0000  0  NULL  NULL
        	            	857106501757796353  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:49:06.288411 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:06.288411 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:06.288411 +0000 +0000  0  NULL  NULL
        	            	857106501985894401  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-16 09:49:06.358027 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:06.358027 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:06.358027 +0000 +0000  0  NULL  NULL
        	            	857106502222905345  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-16 09:49:06.430372 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:06.430372 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:06.430372 +0000 +0000  0  NULL  NULL
        	            	857106502495895553  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-16 09:49:06.513684 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:06.513684 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:06.513684 +0000 +0000  0  NULL  NULL
        	            	857106502742638593  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-16 09:49:06.58897 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:49:06.58897 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:06.58897 +0000 +0000   0  NULL  NULL
        	            	857106502976110593  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-16 09:49:06.660227 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:06.660227 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:06.660227 +0000 +0000  0  NULL  NULL
        	            	857106503214530561  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-16 09:49:06.732986 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:06.732986 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:06.732986 +0000 +0000  0  NULL  NULL
        	            	857106503449477121  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-16 09:49:06.804691 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:06.804691 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:06.804691 +0000 +0000  0  NULL  NULL
        	            	857106503797735425  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-16 09:49:06.910955 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:06.910955 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:06.910955 +0000 +0000  0  NULL  NULL
        	            	857106504036253697  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-16 09:49:06.983764 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:06.983764 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:06.983764 +0000 +0000  0  NULL  NULL
        	            	857106504282046465  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:49:07.058775 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:07.058775 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:07.058775 +0000 +0000  0  NULL  NULL
        	            	857106504519614465  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:49:07.131264 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:07.131264 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:07.131264 +0000 +0000  0  NULL  NULL
        	            	857106504741978113  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:49:07.199124 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:07.199124 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:07.199124 +0000 +0000  0  NULL  NULL
        	            	857106504780840961  AUTO CREATE STATS                Table statistics refresh for system.public.span_stats_tenant_boundaries                 CREATE STATISTICS __auto__ FROM [57] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-16 09:49:07.210965 +0000 +0000  2023-04-16 09:49:07.423809 +0000 +0000  2023-04-16 09:49:07.960011 +0000 +0000  2023-04-16 09:49:07.960068 +0000 +0000  1  NULL    1     5086483068636383124  2023-04-16 09:49:07.423825 +0000 +0000  2023-04-17 09:49:07.423825 +0000 +0000  1  NULL  NULL
        	            	857106505053569025  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:49:07.294205 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:07.294205 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:07.294205 +0000 +0000  0  NULL  NULL
        	            	857106505379282945  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:49:07.39362 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:49:07.39362 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:07.39362 +0000 +0000   0  NULL  NULL
        	            	857106505695166465  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:49:07.490016 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:07.490016 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:07.490016 +0000 +0000  0  NULL  NULL
        	            	857106505939419137  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:49:07.564561 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:07.564561 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:07.564561 +0000 +0000  0  NULL  NULL
        	            	857106506170204161  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:49:07.634988 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:07.634988 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:07.634988 +0000 +0000  0  NULL  NULL
        	            	857106506412589057  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:49:07.708949 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:07.708949 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:07.708949 +0000 +0000  0  NULL  NULL
        	            	857106506648059905  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:49:07.780806 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:07.780806 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:07.780806 +0000 +0000  0  NULL  NULL
        	            	857106506906206209  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:49:07.859585 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:07.859585 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:07.859585 +0000 +0000  0  NULL  NULL
        	            	857106507169529857  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:49:07.939954 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:07.939954 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:07.939954 +0000 +0000  0  NULL  NULL
        	            	857106507410243585  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:49:08.013414 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:08.013414 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:08.013414 +0000 +0000  0  NULL  NULL
        	            	857106507649253377  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:49:08.086357 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:08.086357 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:08.086357 +0000 +0000  0  NULL  NULL
        	            	857106507905302529  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:49:08.164491 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:08.164491 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:08.164491 +0000 +0000  0  NULL  NULL
        	            	857106508157190145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:49:08.241376 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:08.241376 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:08.241376 +0000 +0000  0  NULL  NULL
        	            	857106508422807553  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:49:08.322432 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:08.322432 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:08.322432 +0000 +0000  0  NULL  NULL
        	            	857106508520390657  AUTO CREATE STATS                Table statistics refresh for system.public.transaction_activity                         CREATE STATISTICS __auto__ FROM [61] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-16 09:49:08.352184 +0000 +0000  2023-04-16 09:49:08.516762 +0000 +0000  NULL                                    2023-04-16 09:49:08.459866 +0000 +0000  0  NULL    1     7473480174285982734  2023-04-16 09:49:08.516777 +0000 +0000  2023-04-17 09:49:08.516777 +0000 +0000  1  NULL  NULL
        	            	857106508658016257  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:49:08.394204 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:49:08.394204 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:49:08.394204 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (9.15s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 6cce6c746150307a9ecf3b529dfd633a6985c110:

        	            	857106868959772673  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-16 09:50:58.349571 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:58.349571 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:58.349571 +0000 +0000  0  NULL  NULL
        	            	857106869084880897  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-16 09:50:58.387752 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:58.387752 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:58.387752 +0000 +0000  0  NULL  NULL
        	            	857106869212381185  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-16 09:50:58.42667 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:50:58.42667 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:58.42667 +0000 +0000   0  NULL  NULL
        	            	857106869339586561  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:50:58.465493 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:58.465493 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:58.465493 +0000 +0000  0  NULL  NULL
        	            	857106869416984577  AUTO CREATE STATS                Table statistics refresh for system.public.comments                                     CREATE STATISTICS __auto__ FROM [24] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-16 09:50:58.489086 +0000 +0000  2023-04-16 09:50:58.590927 +0000 +0000  2023-04-16 09:51:00.501028 +0000 +0000  2023-04-16 09:51:00.501137 +0000 +0000  1  NULL    1     8223681224184016729  2023-04-16 09:50:58.590932 +0000 +0000  2023-04-17 09:50:58.590932 +0000 +0000  1  NULL  NULL
        	            	857106869466169345  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:50:58.504132 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:58.504132 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:58.504132 +0000 +0000  0  NULL  NULL
        	            	857106869599404033  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:50:58.544777 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:58.544777 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:58.544777 +0000 +0000  0  NULL  NULL
        	            	857106869739061249  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:50:58.587395 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:58.587395 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:58.587395 +0000 +0000  0  NULL  NULL
        	            	857106869868986369  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:50:58.627036 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:58.627036 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:58.627036 +0000 +0000  0  NULL  NULL
        	            	857106870030532609  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:50:58.676345 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:58.676345 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:58.676345 +0000 +0000  0  NULL  NULL
        	            	857106870156623873  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:50:58.714834 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:58.714834 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:58.714834 +0000 +0000  0  NULL  NULL
        	            	857106870320431105  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:50:58.764814 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:58.764814 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:58.764814 +0000 +0000  0  NULL  NULL
        	            	857106870462939137  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:50:58.80831 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:50:58.80831 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:58.80831 +0000 +0000   0  NULL  NULL
        	            	857106870621798401  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:50:58.85678 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:50:58.85678 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:58.85678 +0000 +0000   0  NULL  NULL
        	            	857106870806249473  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:50:58.913073 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:58.913073 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:58.913073 +0000 +0000  0  NULL  NULL
        	            	857106870934536193  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:50:58.952232 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:58.952232 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:58.952232 +0000 +0000  0  NULL  NULL
        	            	857106871068098561  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:50:58.992992 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:58.992992 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:58.992992 +0000 +0000  0  NULL  NULL
        	            	857106871226761217  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:50:59.041405 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.041405 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.041405 +0000 +0000  0  NULL  NULL
        	            	857106871359045633  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:50:59.081781 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.081781 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.081781 +0000 +0000  0  NULL  NULL
        	            	857106871490478081  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:50:59.121887 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.121887 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.121887 +0000 +0000  0  NULL  NULL
        	            	857106871620108289  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:50:59.161448 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.161448 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.161448 +0000 +0000  0  NULL  NULL
        	            	857106871749967873  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:50:59.201074 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.201074 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.201074 +0000 +0000  0  NULL  NULL
        	            	857106871878057985  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-16 09:50:59.240182 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.240182 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.240182 +0000 +0000  0  NULL  NULL
        	            	857106872012767233  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-16 09:50:59.281277 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.281277 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.281277 +0000 +0000  0  NULL  NULL
        	            	857106872144166913  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-16 09:50:59.321375 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.321375 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.321375 +0000 +0000  0  NULL  NULL
        	            	857106872275927041  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-16 09:50:59.361585 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.361585 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.361585 +0000 +0000  0  NULL  NULL
        	            	857106872401035265  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-16 09:50:59.39977 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:50:59.39977 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.39977 +0000 +0000   0  NULL  NULL
        	            	857106872563630081  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-16 09:50:59.449389 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.449389 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.449389 +0000 +0000  0  NULL  NULL
        	            	857106872686313473  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-16 09:50:59.486825 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.486825 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.486825 +0000 +0000  0  NULL  NULL
        	            	857106872883281921  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-16 09:50:59.546941 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.546941 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.546941 +0000 +0000  0  NULL  NULL
        	            	857106873009307649  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-16 09:50:59.585405 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.585405 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.585405 +0000 +0000  0  NULL  NULL
        	            	857106873136545793  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:50:59.624223 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.624223 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.624223 +0000 +0000  0  NULL  NULL
        	            	857106873261850625  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:50:59.66246 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:50:59.66246 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.66246 +0000 +0000   0  NULL  NULL
        	            	857106873388236801  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:50:59.701031 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.701031 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.701031 +0000 +0000  0  NULL  NULL
        	            	857106873537200129  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:50:59.746497 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.746497 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.746497 +0000 +0000  0  NULL  NULL
        	            	857106873674760193  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:50:59.788492 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.788492 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.788492 +0000 +0000  0  NULL  NULL
        	            	857106873800425473  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:50:59.826829 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.826829 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.826829 +0000 +0000  0  NULL  NULL
        	            	857106873927139329  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:50:59.865495 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.865495 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.865495 +0000 +0000  0  NULL  NULL
        	            	857106874051002369  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:50:59.903295 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.903295 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.903295 +0000 +0000  0  NULL  NULL
        	            	857106874322386945  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:50:59.986175 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:50:59.986175 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:50:59.986175 +0000 +0000  0  NULL  NULL
        	            	857106874469023745  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:51:00.030883 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:51:00.030883 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:51:00.030883 +0000 +0000  0  NULL  NULL
        	            	857106874609369089  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:51:00.073707 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:51:00.073707 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:51:00.073707 +0000 +0000  0  NULL  NULL
        	            	857106874776387585  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:51:00.12467 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:51:00.12467 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:51:00.12467 +0000 +0000   0  NULL  NULL
        	            	857106874933084161  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:51:00.172493 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:51:00.172493 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:51:00.172493 +0000 +0000  0  NULL  NULL
        	            	857106875122352129  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:51:00.230258 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:51:00.230258 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:51:00.230258 +0000 +0000  0  NULL  NULL
        	            	857106875272134657  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:51:00.275977 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:51:00.275977 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:51:00.275977 +0000 +0000  0  NULL  NULL
        	            	857106875434991617  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:51:00.325662 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:51:00.325662 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:51:00.325662 +0000 +0000  0  NULL  NULL
        	            	857106875597127681  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:51:00.375138 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:51:00.375138 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:51:00.375138 +0000 +0000  0  NULL  NULL
        	            	857106875755397121  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:51:00.423431 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:51:00.423431 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:51:00.423431 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (7.47s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 6cce6c746150307a9ecf3b529dfd633a6985c110:

        	            	857389825792671745  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-17 09:50:09.922113 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:09.922113 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:09.922113 +0000 +0000  0  NULL  NULL
        	            	857389826054258689  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-17 09:50:10.001943 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:10.001943 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:10.001943 +0000 +0000  0  NULL  NULL
        	            	857389826290712577  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 09:50:10.074107 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:10.074107 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:10.074107 +0000 +0000  0  NULL  NULL
        	            	857389826546008065  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 09:50:10.152023 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:10.152023 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:10.152023 +0000 +0000  0  NULL  NULL
        	            	857389826784985089  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 09:50:10.224942 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:10.224942 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:10.224942 +0000 +0000  0  NULL  NULL
        	            	857389827007905793  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 09:50:10.292991 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:10.292991 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:10.292991 +0000 +0000  0  NULL  NULL
        	            	857389827232727041  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 09:50:10.361578 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:10.361578 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:10.361578 +0000 +0000  0  NULL  NULL
        	            	857389827486351361  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 09:50:10.43898 +0000 +0000   NULL                                    NULL                                    2023-04-17 09:50:10.43898 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:10.43898 +0000 +0000   0  NULL  NULL
        	            	857389827710353409  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 09:50:10.507337 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:10.507337 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:10.507337 +0000 +0000  0  NULL  NULL
        	            	857389827956932609  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 09:50:10.582591 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:10.582591 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:10.582591 +0000 +0000  0  NULL  NULL
        	            	857389828185587713  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 09:50:10.652367 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:10.652367 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:10.652367 +0000 +0000  0  NULL  NULL
        	            	857389828413489153  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 09:50:10.721911 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:10.721911 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:10.721911 +0000 +0000  0  NULL  NULL
        	            	857389828632903681  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 09:50:10.788876 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:10.788876 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:10.788876 +0000 +0000  0  NULL  NULL
        	            	857389828855922689  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 09:50:10.856932 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:10.856932 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:10.856932 +0000 +0000  0  NULL  NULL
        	            	857389829108137985  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 09:50:10.933906 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:10.933906 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:10.933906 +0000 +0000  0  NULL  NULL
        	            	857389829353046017  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 09:50:11.008655 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:11.008655 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:11.008655 +0000 +0000  0  NULL  NULL
        	            	857389829596282881  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 09:50:11.082889 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:11.082889 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:11.082889 +0000 +0000  0  NULL  NULL
        	            	857389829856362497  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 09:50:11.162262 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:11.162262 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:11.162262 +0000 +0000  0  NULL  NULL
        	            	857389830082166785  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 09:50:11.231162 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:11.231162 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:11.231162 +0000 +0000  0  NULL  NULL
        	            	857389830290604033  AUTO CREATE STATS                Table statistics refresh for system.public.transaction_statistics                       CREATE STATISTICS __auto__ FROM [43] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-17 09:50:11.294757 +0000 +0000  2023-04-17 09:50:11.450289 +0000 +0000  2023-04-17 09:50:12.831266 +0000 +0000  2023-04-17 09:50:12.831338 +0000 +0000  1  NULL    1     115118085478147012   2023-04-17 09:50:11.450327 +0000 +0000  2023-04-18 09:50:11.450327 +0000 +0000  1  NULL  NULL
        	            	857389830319996929  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 09:50:11.303737 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:11.303737 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:11.303737 +0000 +0000  0  NULL  NULL
        	            	857389830558744577  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-17 09:50:11.376597 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:11.376597 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:11.376597 +0000 +0000  0  NULL  NULL
        	            	857389830794936321  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-17 09:50:11.448683 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:11.448683 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:11.448683 +0000 +0000  0  NULL  NULL
        	            	857389831012122625  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-17 09:50:11.514971 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:11.514971 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:11.514971 +0000 +0000  0  NULL  NULL
        	            	857389831240581121  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-17 09:50:11.584688 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:11.584688 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:11.584688 +0000 +0000  0  NULL  NULL
        	            	857389831464386561  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-17 09:50:11.653 +0000 +0000     NULL                                    NULL                                    2023-04-17 09:50:11.653 +0000 +0000     0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:11.653 +0000 +0000     0  NULL  NULL
        	            	857389831692189697  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-17 09:50:11.722498 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:11.722498 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:11.722498 +0000 +0000  0  NULL  NULL
        	            	857389831918452737  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-17 09:50:11.791545 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:11.791545 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:11.791545 +0000 +0000  0  NULL  NULL
        	            	857389832199503873  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-17 09:50:11.87732 +0000 +0000   NULL                                    NULL                                    2023-04-17 09:50:11.87732 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:11.87732 +0000 +0000   0  NULL  NULL
        	            	857389832422490113  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-17 09:50:11.945369 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:11.945369 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:11.945369 +0000 +0000  0  NULL  NULL
        	            	857389832656355329  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 09:50:12.01674 +0000 +0000   NULL                                    NULL                                    2023-04-17 09:50:12.01674 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:12.01674 +0000 +0000   0  NULL  NULL
        	            	857389832884977665  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 09:50:12.086515 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:12.086515 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:12.086515 +0000 +0000  0  NULL  NULL
        	            	857389833120284673  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 09:50:12.158314 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:12.158314 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:12.158314 +0000 +0000  0  NULL  NULL
        	            	857389833371746305  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 09:50:12.235064 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:12.235064 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:12.235064 +0000 +0000  0  NULL  NULL
        	            	857389833593782273  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 09:50:12.302818 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:12.302818 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:12.302818 +0000 +0000  0  NULL  NULL
        	            	857389833836298241  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 09:50:12.376832 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:12.376832 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:12.376832 +0000 +0000  0  NULL  NULL
        	            	857389834070261761  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 09:50:12.448234 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:12.448234 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:12.448234 +0000 +0000  0  NULL  NULL
        	            	857389834295902209  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 09:50:12.517085 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:12.517085 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:12.517085 +0000 +0000  0  NULL  NULL
        	            	857389834561650689  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 09:50:12.598199 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:12.598199 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:12.598199 +0000 +0000  0  NULL  NULL
        	            	857389834796826625  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 09:50:12.669957 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:12.669957 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:12.669957 +0000 +0000  0  NULL  NULL
        	            	857389835030790145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 09:50:12.741355 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:12.741355 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:12.741355 +0000 +0000  0  NULL  NULL
        	            	857389835276550145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 09:50:12.816364 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:12.816364 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:12.816364 +0000 +0000  0  NULL  NULL
        	            	857389835498455041  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 09:50:12.884084 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:12.884084 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:12.884084 +0000 +0000  0  NULL  NULL
        	            	857389835722522625  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 09:50:12.952459 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:12.952459 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:12.952459 +0000 +0000  0  NULL  NULL
        	            	857389835955830785  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 09:50:13.023658 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:13.023658 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:13.023658 +0000 +0000  0  NULL  NULL
        	            	857389836187140097  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 09:50:13.09426 +0000 +0000   NULL                                    NULL                                    2023-04-17 09:50:13.09426 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:13.09426 +0000 +0000   0  NULL  NULL
        	            	857389836431884289  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 09:50:13.168934 +0000 +0000  NULL                                    NULL                                    2023-04-17 09:50:13.168934 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:13.168934 +0000 +0000  0  NULL  NULL
        	            	857389836463898625  AUTO CREATE STATS                Table statistics refresh for system.public.tenant_settings                              CREATE STATISTICS __auto__ FROM [50] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-17 09:50:13.178695 +0000 +0000  2023-04-17 09:50:13.374593 +0000 +0000  NULL                                    2023-04-17 09:50:13.29324 +0000 +0000   0  NULL    1     7567915845899565580  2023-04-17 09:50:13.374607 +0000 +0000  2023-04-18 09:50:13.374607 +0000 +0000  1  NULL  NULL
        	            	857389836710019073  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 09:50:13.25382 +0000 +0000   NULL                                    NULL                                    2023-04-17 09:50:13.25382 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 09:50:13.25382 +0000 +0000   0  NULL  NULL
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (8.76s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 6cce6c746150307a9ecf3b529dfd633a6985c110:

        	            	857396009184165889  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-17 10:21:36.943459 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:36.943459 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:36.943459 +0000 +0000  0  NULL  NULL
        	            	857396009325199361  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-17 10:21:36.9865 +0000 +0000    NULL                                    NULL                                    2023-04-17 10:21:36.9865 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:36.9865 +0000 +0000    0  NULL  NULL
        	            	857396009460269057  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-17 10:21:37.027699 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.027699 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.027699 +0000 +0000  0  NULL  NULL
        	            	857396009614245889  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 10:21:37.074694 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.074694 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.074694 +0000 +0000  0  NULL  NULL
        	            	857396009777332225  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 10:21:37.124474 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.124474 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.124474 +0000 +0000  0  NULL  NULL
        	            	857396009944055809  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 10:21:37.175339 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.175339 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.175339 +0000 +0000  0  NULL  NULL
        	            	857396010081091585  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 10:21:37.217174 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.217174 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.217174 +0000 +0000  0  NULL  NULL
        	            	857396010216685569  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 10:21:37.258547 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.258547 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.258547 +0000 +0000  0  NULL  NULL
        	            	857396010353328129  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 10:21:37.300245 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.300245 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.300245 +0000 +0000  0  NULL  NULL
        	            	857396010484727809  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 10:21:37.340344 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.340344 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.340344 +0000 +0000  0  NULL  NULL
        	            	857396010616258561  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 10:21:37.380489 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.380489 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.380489 +0000 +0000  0  NULL  NULL
        	            	857396010787209217  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 10:21:37.432651 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.432651 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.432651 +0000 +0000  0  NULL  NULL
        	            	857396010934304769  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 10:21:37.477548 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.477548 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.477548 +0000 +0000  0  NULL  NULL
        	            	857396011067047937  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 10:21:37.518067 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.518067 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.518067 +0000 +0000  0  NULL  NULL
        	            	857396011212439553  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 10:21:37.562452 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.562452 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.562452 +0000 +0000  0  NULL  NULL
        	            	857396011540480001  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 10:21:37.662553 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.662553 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.662553 +0000 +0000  0  NULL  NULL
        	            	857396011718148097  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 10:21:37.716756 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.716756 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.716756 +0000 +0000  0  NULL  NULL
        	            	857396011881627649  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 10:21:37.766661 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.766661 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.766661 +0000 +0000  0  NULL  NULL
        	            	857396012023808001  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 10:21:37.810059 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.810059 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.810059 +0000 +0000  0  NULL  NULL
        	            	857396012212191233  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 10:21:37.867526 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.867526 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.867526 +0000 +0000  0  NULL  NULL
        	            	857396012364398593  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 10:21:37.913981 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:37.913981 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.913981 +0000 +0000  0  NULL  NULL
        	            	857396012508250113  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-17 10:21:37.95791 +0000 +0000   NULL                                    NULL                                    2023-04-17 10:21:37.95791 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:37.95791 +0000 +0000   0  NULL  NULL
        	            	857396012704595969  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-17 10:21:38.017797 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:38.017797 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.017797 +0000 +0000  0  NULL  NULL
        	            	857396012844056577  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-17 10:21:38.06038 +0000 +0000   NULL                                    NULL                                    2023-04-17 10:21:38.06038 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.06038 +0000 +0000   0  NULL  NULL
        	            	857396013065863169  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-17 10:21:38.128062 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:38.128062 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.128062 +0000 +0000  0  NULL  NULL
        	            	857396013253689345  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-17 10:21:38.185375 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:38.185375 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.185375 +0000 +0000  0  NULL  NULL
        	            	857396013419659265  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-17 10:21:38.236021 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:38.236021 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.236021 +0000 +0000  0  NULL  NULL
        	            	857396013555286017  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-17 10:21:38.277406 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:38.277406 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.277406 +0000 +0000  0  NULL  NULL
        	            	857396013695074305  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-17 10:21:38.320093 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:38.320093 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.320093 +0000 +0000  0  NULL  NULL
        	            	857396013853081601  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-17 10:21:38.368293 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:38.368293 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.368293 +0000 +0000  0  NULL  NULL
        	            	857396013987987457  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 10:21:38.409462 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:38.409462 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.409462 +0000 +0000  0  NULL  NULL
        	            	857396014126039041  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 10:21:38.451615 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:38.451615 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.451615 +0000 +0000  0  NULL  NULL
        	            	857396014276313089  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 10:21:38.497458 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:38.497458 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.497458 +0000 +0000  0  NULL  NULL
        	            	857396014421639169  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 10:21:38.541801 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:38.541801 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.541801 +0000 +0000  0  NULL  NULL
        	            	857396014564638721  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 10:21:38.585438 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:38.585438 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.585438 +0000 +0000  0  NULL  NULL
        	            	857396014707212289  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 10:21:38.628956 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:38.628956 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.628956 +0000 +0000  0  NULL  NULL
        	            	857396014854701057  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 10:21:38.673972 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:38.673972 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.673972 +0000 +0000  0  NULL  NULL
        	            	857396015047901185  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 10:21:38.732915 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:38.732915 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.732915 +0000 +0000  0  NULL  NULL
        	            	857396015189983233  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 10:21:38.776284 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:38.776284 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.776284 +0000 +0000  0  NULL  NULL
        	            	857396015369420801  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 10:21:38.831054 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:38.831054 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.831054 +0000 +0000  0  NULL  NULL
        	            	857396015618195457  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 10:21:38.90696 +0000 +0000   NULL                                    NULL                                    2023-04-17 10:21:38.90696 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.90696 +0000 +0000   0  NULL  NULL
        	            	857396015867068417  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 10:21:38.982909 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:38.982909 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:38.982909 +0000 +0000  0  NULL  NULL
        	            	857396016169713665  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 10:21:39.075273 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:39.075273 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:39.075273 +0000 +0000  0  NULL  NULL
        	            	857396016326541313  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 10:21:39.123129 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:39.123129 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:39.123129 +0000 +0000  0  NULL  NULL
        	            	857396016470491137  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 10:21:39.167058 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:39.167058 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:39.167058 +0000 +0000  0  NULL  NULL
        	            	857396016660807681  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 10:21:39.225147 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:39.225147 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:39.225147 +0000 +0000  0  NULL  NULL
        	            	857396016812425217  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 10:21:39.271422 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:39.271422 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:39.271422 +0000 +0000  0  NULL  NULL
        	            	857396016967057409  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 10:21:39.318599 +0000 +0000  NULL                                    NULL                                    2023-04-17 10:21:39.318599 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 10:21:39.318599 +0000 +0000  0  NULL  NULL
        	            	857396017105305601  AUTO CREATE STATS                Table statistics refresh for system.public.web_sessions                                 CREATE STATISTICS __auto__ FROM [19] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-17 10:21:39.360766 +0000 +0000  2023-04-17 10:21:39.491977 +0000 +0000  NULL                                    2023-04-17 10:21:39.452991 +0000 +0000  0  NULL    1     4172408922916262069  2023-04-17 10:21:39.491982 +0000 +0000  2023-04-18 10:21:39.491982 +0000 +0000  1  NULL  NULL
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (8.58s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 12607684868d04f957b3385f142e4e8b9b55c799:

        	            	857680307697287169  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 10:27:37.964284 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:37.964284 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:37.964284 +0000 +0000  0  NULL  NULL
        	            	857680307960217601  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 10:27:38.044523 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:38.044523 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:38.044523 +0000 +0000  0  NULL  NULL
        	            	857680308217839617  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 10:27:38.123148 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:38.123148 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:38.123148 +0000 +0000  0  NULL  NULL
        	            	857680308455211009  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 10:27:38.195577 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:38.195577 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:38.195577 +0000 +0000  0  NULL  NULL
        	            	857680308680294401  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-18 10:27:38.264287 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:38.264287 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:38.264287 +0000 +0000  0  NULL  NULL
        	            	857680308946141185  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-18 10:27:38.345413 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:38.345413 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:38.345413 +0000 +0000  0  NULL  NULL
        	            	857680309213298689  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-18 10:27:38.426935 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:38.426935 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:38.426935 +0000 +0000  0  NULL  NULL
        	            	857680309457551361  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 10:27:38.501475 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:38.501475 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:38.501475 +0000 +0000  0  NULL  NULL
        	            	857680309685747713  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 10:27:38.571127 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:38.571127 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:38.571127 +0000 +0000  0  NULL  NULL
        	            	857680309933539329  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 10:27:38.646739 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:38.646739 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:38.646739 +0000 +0000  0  NULL  NULL
        	            	857680310181298177  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 10:27:38.722345 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:38.722345 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:38.722345 +0000 +0000  0  NULL  NULL
        	            	857680310374236161  AUTO CREATE STATS                Table statistics refresh for system.public.role_members                                 CREATE STATISTICS __auto__ FROM [23] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-18 10:27:38.781213 +0000 +0000  2023-04-18 10:27:38.941263 +0000 +0000  2023-04-18 10:27:39.73468 +0000 +0000   2023-04-18 10:27:39.734735 +0000 +0000  1  NULL    1     761578764018996064   2023-04-18 10:27:38.941326 +0000 +0000  2023-04-19 10:27:38.941326 +0000 +0000  1  NULL  NULL
        	            	857680310413393921  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 10:27:38.793188 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:38.793188 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:38.793188 +0000 +0000  0  NULL  NULL
        	            	857680310659252225  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 10:27:38.868214 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:38.868214 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:38.868214 +0000 +0000  0  NULL  NULL
        	            	857680310911369217  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 10:27:38.945142 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:38.945142 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:38.945142 +0000 +0000  0  NULL  NULL
        	            	857680311160864769  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 10:27:39.021293 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:39.021293 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:39.021293 +0000 +0000  0  NULL  NULL
        	            	857680311412719617  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 10:27:39.098144 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:39.098144 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:39.098144 +0000 +0000  0  NULL  NULL
        	            	857680311683383297  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:27:39.180757 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:39.180757 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:39.180757 +0000 +0000  0  NULL  NULL
        	            	857680311950835713  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:27:39.262375 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:39.262375 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:39.262375 +0000 +0000  0  NULL  NULL
        	            	857680312214454273  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:27:39.342816 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:39.342816 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:39.342816 +0000 +0000  0  NULL  NULL
        	            	857680312496685057  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:27:39.428944 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:42.553225 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:39.428944 +0000 +0000  0  NULL  NULL
        	            	857680312760958977  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:27:39.509667 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:42.638091 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:39.509667 +0000 +0000  0  NULL  NULL
        	            	857680313008586753  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:27:39.585165 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:42.731703 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:39.585165 +0000 +0000  0  NULL  NULL
        	            	857680313242058753  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:27:39.65642 +0000 +0000   NULL                                    NULL                                    2023-04-18 10:27:43.141229 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:39.65642 +0000 +0000   0  NULL  NULL
        	            	857680313502892033  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:27:39.736022 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:43.230999 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:39.736022 +0000 +0000  0  NULL  NULL
        	            	857680313737445377  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:27:39.807603 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:43.322164 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:39.807603 +0000 +0000  0  NULL  NULL
        	            	857680313979437057  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 10:27:39.881456 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:39.881456 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:39.881456 +0000 +0000  0  NULL  NULL
        	            	857680314226343937  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 10:27:39.956803 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:39.956803 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:39.956803 +0000 +0000  0  NULL  NULL
        	            	857680314465157121  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 10:27:40.029681 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:40.029681 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:40.029681 +0000 +0000  0  NULL  NULL
        	            	857680314707640321  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 10:27:40.10368 +0000 +0000   NULL                                    NULL                                    2023-04-18 10:27:40.10368 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:40.10368 +0000 +0000   0  NULL  NULL
        	            	857680314940227585  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 10:27:40.174655 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:40.174655 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:40.174655 +0000 +0000  0  NULL  NULL
        	            	857680315163443201  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 10:27:40.242772 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:40.242772 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:40.242772 +0000 +0000  0  NULL  NULL
        	            	857680315365752833  AUTO CREATE STATS                Table statistics refresh for system.public.span_configurations                          CREATE STATISTICS __auto__ FROM [47] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-18 10:27:40.304507 +0000 +0000  2023-04-18 10:27:40.460095 +0000 +0000  2023-04-18 10:27:41.055491 +0000 +0000  2023-04-18 10:27:41.055545 +0000 +0000  1  NULL    1     8256629746043906994  2023-04-18 10:27:40.460119 +0000 +0000  2023-04-19 10:27:40.460119 +0000 +0000  1  NULL  NULL
        	            	857680315405697025  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:27:40.316721 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:42.871281 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:40.316721 +0000 +0000  0  NULL  NULL
        	            	857680315641200641  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:27:40.388573 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:42.958422 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:40.388573 +0000 +0000  0  NULL  NULL
        	            	857680315896332289  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:27:40.466439 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:43.046763 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:40.466439 +0000 +0000  0  NULL  NULL
        	            	857680316138029057  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 10:27:40.540208 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:40.540208 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:40.540208 +0000 +0000  0  NULL  NULL
        	            	857680316390703105  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 10:27:40.617314 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:40.617314 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:40.617314 +0000 +0000  0  NULL  NULL
        	            	857680316667035649  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 10:27:40.701647 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:40.701647 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:40.701647 +0000 +0000  0  NULL  NULL
        	            	857680316922232833  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 10:27:40.779524 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:40.779524 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:40.779524 +0000 +0000  0  NULL  NULL
        	            	857680317172219905  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 10:27:40.855814 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:40.855814 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:40.855814 +0000 +0000  0  NULL  NULL
        	            	857680317409624065  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 10:27:40.928258 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:40.928258 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:40.928258 +0000 +0000  0  NULL  NULL
        	            	857680317643784193  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 10:27:40.999719 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:40.999719 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:40.999719 +0000 +0000  0  NULL  NULL
        	            	857680317902225409  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 10:27:41.078593 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:41.078593 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:41.078593 +0000 +0000  0  NULL  NULL
        	            	857680318137860097  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 10:27:41.150496 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:41.150496 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:41.150496 +0000 +0000  0  NULL  NULL
        	            	857680318374019073  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:27:41.222568 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:41.222568 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:41.222568 +0000 +0000  0  NULL  NULL
        	            	857680318610964481  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:27:41.294877 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:41.294877 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:41.294877 +0000 +0000  0  NULL  NULL
        	            	857680318842208257  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:27:41.365459 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:27:41.365459 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:27:41.365459 +0000 +0000  0  NULL  NULL
        	            	857680318925766657  AUTO CREATE STATS                Table statistics refresh for system.public.descriptor                                   CREATE STATISTICS __auto__ FROM [3] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'   root      {}  running           NULL  2023-04-18 10:27:41.390934 +0000 +0000  2023-04-18 10:27:41.532832 +0000 +0000  NULL                                    2023-04-18 10:27:41.483886 +0000 +0000  0  NULL    1     7334157449449521518  2023-04-18 10:27:41.532842 +0000 +0000  2023-04-19 10:27:41.532842 +0000 +0000  1  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (10.60s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 12607684868d04f957b3385f142e4e8b9b55c799:

        	            	857680827971764225  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 10:30:16.739468 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:16.739468 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:16.739468 +0000 +0000  0  NULL  NULL
        	            	857680828115845121  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 10:30:16.783459 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:16.783459 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:16.783459 +0000 +0000  0  NULL  NULL
        	            	857680828270280705  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 10:30:16.830565 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:16.830565 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:16.830565 +0000 +0000  0  NULL  NULL
        	            	857680828422750209  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 10:30:16.877095 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:16.877095 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:16.877095 +0000 +0000  0  NULL  NULL
        	            	857680828572827649  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 10:30:16.922905 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:16.922905 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:16.922905 +0000 +0000  0  NULL  NULL
        	            	857680828729131009  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 10:30:16.970602 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:16.970602 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:16.970602 +0000 +0000  0  NULL  NULL
        	            	857680828884844545  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-18 10:30:17.018133 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:17.018133 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.018133 +0000 +0000  0  NULL  NULL
        	            	857680829032366081  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-18 10:30:17.063143 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:17.063143 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.063143 +0000 +0000  0  NULL  NULL
        	            	857680829205086209  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-18 10:30:17.115855 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:17.115855 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.115855 +0000 +0000  0  NULL  NULL
        	            	857680829364436993  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 10:30:17.164479 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:17.164479 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.164479 +0000 +0000  0  NULL  NULL
        	            	857680829509369857  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 10:30:17.208718 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:17.208718 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.208718 +0000 +0000  0  NULL  NULL
        	            	857680829665902593  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 10:30:17.25649 +0000 +0000   NULL                                    NULL                                    2023-04-18 10:30:17.25649 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.25649 +0000 +0000   0  NULL  NULL
        	            	857680829747789825  AUTO CREATE STATS                Table statistics refresh for system.public.role_members                                 CREATE STATISTICS __auto__ FROM [23] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-18 10:30:17.281468 +0000 +0000  2023-04-18 10:30:17.405731 +0000 +0000  2023-04-18 10:30:19.125009 +0000 +0000  2023-04-18 10:30:19.125163 +0000 +0000  1  NULL    1     6459950034797176086  2023-04-18 10:30:17.405737 +0000 +0000  2023-04-19 10:30:17.405737 +0000 +0000  1  NULL  NULL
        	            	857680829842259969  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 10:30:17.31029 +0000 +0000   NULL                                    NULL                                    2023-04-18 10:30:17.31029 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.31029 +0000 +0000   0  NULL  NULL
        	            	857680829999972353  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 10:30:17.358424 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:17.358424 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.358424 +0000 +0000  0  NULL  NULL
        	            	857680830138646529  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 10:30:17.400753 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:17.400753 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.400753 +0000 +0000  0  NULL  NULL
        	            	857680830283644929  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 10:30:17.445009 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:17.445009 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.445009 +0000 +0000  0  NULL  NULL
        	            	857680830455382017  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 10:30:17.497404 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:17.497404 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.497404 +0000 +0000  0  NULL  NULL
        	            	857680830613684225  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 10:30:17.545716 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:17.545716 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.545716 +0000 +0000  0  NULL  NULL
        	            	857680830792794113  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:30:17.600398 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:17.600398 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.600398 +0000 +0000  0  NULL  NULL
        	            	857680830948507649  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:30:17.647921 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:17.647921 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.647921 +0000 +0000  0  NULL  NULL
        	            	857680831089508353  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:30:17.690935 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:17.690935 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.690935 +0000 +0000  0  NULL  NULL
        	            	857680831246237697  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:30:17.738758 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:21.472596 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.738758 +0000 +0000  0  NULL  NULL
        	            	857680831387959297  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:30:17.782011 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:21.54176 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.782011 +0000 +0000  0  NULL  NULL
        	            	857680831549407233  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:30:17.831277 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:21.624238 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.831277 +0000 +0000  0  NULL  NULL
        	            	857680831685165057  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:30:17.87271 +0000 +0000   NULL                                    NULL                                    2023-04-18 10:30:22.370507 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.87271 +0000 +0000   0  NULL  NULL
        	            	857680831823446017  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:30:17.914922 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:22.444786 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.914922 +0000 +0000  0  NULL  NULL
        	            	857680831995478017  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:30:17.967426 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:22.520416 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:17.967426 +0000 +0000  0  NULL  NULL
        	            	857680832128483329  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 10:30:18.008006 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:18.008006 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.008006 +0000 +0000  0  NULL  NULL
        	            	857680832266895361  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 10:30:18.050247 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:18.050247 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.050247 +0000 +0000  0  NULL  NULL
        	            	857680832403931137  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 10:30:18.092065 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:18.092065 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.092065 +0000 +0000  0  NULL  NULL
        	            	857680832572653569  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 10:30:18.143547 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:18.143547 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.143547 +0000 +0000  0  NULL  NULL
        	            	857680832726663169  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 10:30:18.19056 +0000 +0000   NULL                                    NULL                                    2023-04-18 10:30:18.19056 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.19056 +0000 +0000   0  NULL  NULL
        	            	857680832872710145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 10:30:18.235118 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:18.235118 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.235118 +0000 +0000  0  NULL  NULL
        	            	857680833010204673  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:30:18.277081 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:22.139997 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.277081 +0000 +0000  0  NULL  NULL
        	            	857680833157300225  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:30:18.321965 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:22.209385 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.321965 +0000 +0000  0  NULL  NULL
        	            	857680833303674881  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:30:18.366639 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:22.283437 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.366639 +0000 +0000  0  NULL  NULL
        	            	857680833440022529  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 10:30:18.408248 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:18.408248 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.408248 +0000 +0000  0  NULL  NULL
        	            	857680833577123841  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 10:30:18.450089 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:18.450089 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.450089 +0000 +0000  0  NULL  NULL
        	            	857680833712947201  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 10:30:18.491559 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:18.491559 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.491559 +0000 +0000  0  NULL  NULL
        	            	857680833851457537  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 10:30:18.533812 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:18.533812 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.533812 +0000 +0000  0  NULL  NULL
        	            	857680833991344129  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 10:30:18.576503 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:18.576503 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.576503 +0000 +0000  0  NULL  NULL
        	            	857680834133262337  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 10:30:18.619816 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:18.619816 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.619816 +0000 +0000  0  NULL  NULL
        	            	857680834292973569  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 10:30:18.66855 +0000 +0000   NULL                                    NULL                                    2023-04-18 10:30:18.66855 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.66855 +0000 +0000   0  NULL  NULL
        	            	857680834459664385  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 10:30:18.719435 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:18.719435 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.719435 +0000 +0000  0  NULL  NULL
        	            	857680834616688641  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 10:30:18.767342 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:18.767342 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.767342 +0000 +0000  0  NULL  NULL
        	            	857680834760474625  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:30:18.811216 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:18.811216 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.811216 +0000 +0000  0  NULL  NULL
        	            	857680834901409793  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:30:18.854233 +0000 +0000  NULL                                    NULL                                    2023-04-18 10:30:18.854233 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.854233 +0000 +0000  0  NULL  NULL
        	            	857680835042607105  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 10:30:18.89733 +0000 +0000   NULL                                    NULL                                    2023-04-18 10:30:18.89733 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-18 10:30:18.89733 +0000 +0000   0  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (10.42s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 2f86475f2e3d713b4c8c976c34dba99acce7c4c7:

        	            	857952506623393793  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-19 09:32:06.484212 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:06.484212 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:06.484212 +0000 +0000  0  NULL  NULL
        	            	857952506866892801  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-19 09:32:06.558531 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:06.558531 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:06.558531 +0000 +0000  0  NULL  NULL
        	            	857952507096334337  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-19 09:32:06.628552 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:06.628552 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:06.628552 +0000 +0000  0  NULL  NULL
        	            	857952507330101249  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-19 09:32:06.699888 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:06.699888 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:06.699888 +0000 +0000  0  NULL  NULL
        	            	857952507566194689  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-19 09:32:06.771936 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:06.771936 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:06.771936 +0000 +0000  0  NULL  NULL
        	            	857952507824603137  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-19 09:32:06.850823 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:06.850823 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:06.850823 +0000 +0000  0  NULL  NULL
        	            	857952508095692801  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-19 09:32:06.933528 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:06.933528 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:06.933528 +0000 +0000  0  NULL  NULL
        	            	857952508337225729  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-19 09:32:07.007238 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:07.007238 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:07.007238 +0000 +0000  0  NULL  NULL
        	            	857952508613165057  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-19 09:32:07.091451 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:07.091451 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:07.091451 +0000 +0000  0  NULL  NULL
        	            	857952508844802049  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-19 09:32:07.162135 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:07.162135 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:07.162135 +0000 +0000  0  NULL  NULL
        	            	857952509113368577  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-19 09:32:07.2441 +0000 +0000    NULL                                    NULL                                    2023-04-19 09:32:07.2441 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:07.2441 +0000 +0000    0  NULL  NULL
        	            	857952509224714241  AUTO CREATE STATS                Table statistics refresh for system.public.protected_ts_records                         CREATE STATISTICS __auto__ FROM [32] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-19 09:32:07.278061 +0000 +0000  2023-04-19 09:32:07.439511 +0000 +0000  2023-04-19 09:32:08.205288 +0000 +0000  2023-04-19 09:32:08.205341 +0000 +0000  1  NULL    1     757142187401227273   2023-04-19 09:32:07.439527 +0000 +0000  2023-04-20 09:32:07.439527 +0000 +0000  1  NULL  NULL
        	            	857952509362044929  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-19 09:32:07.319991 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:07.319991 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:07.319991 +0000 +0000  0  NULL  NULL
        	            	857952509611769857  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-19 09:32:07.396198 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:07.396198 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:07.396198 +0000 +0000  0  NULL  NULL
        	            	857952509842718721  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-19 09:32:07.466669 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:07.466669 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:07.466669 +0000 +0000  0  NULL  NULL
        	            	857952510065475585  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-19 09:32:07.534663 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:07.534663 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:07.534663 +0000 +0000  0  NULL  NULL
        	            	857952510297997313  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-19 09:32:07.60562 +0000 +0000   NULL                                    NULL                                    2023-04-19 09:32:07.60562 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:07.60562 +0000 +0000   0  NULL  NULL
        	            	857952510532255745  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:32:07.677102 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:07.677102 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:07.677102 +0000 +0000  0  NULL  NULL
        	            	857952510764220417  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:32:07.747898 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:07.747898 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:07.747898 +0000 +0000  0  NULL  NULL
        	            	857952510995955713  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:32:07.818615 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:07.818615 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:07.818615 +0000 +0000  0  NULL  NULL
        	            	857952511233196033  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:32:07.891028 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:11.072128 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:07.891028 +0000 +0000  0  NULL  NULL
        	            	857952511463358465  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:32:07.961267 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:11.15524 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:07.961267 +0000 +0000  0  NULL  NULL
        	            	857952511738150913  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:32:08.04512 +0000 +0000   NULL                                    NULL                                    2023-04-19 09:32:11.234816 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:08.04512 +0000 +0000   0  NULL  NULL
        	            	857952511978799105  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:32:08.118558 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:11.628726 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:08.118558 +0000 +0000  0  NULL  NULL
        	            	857952512211386369  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:32:08.189576 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:11.712033 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:08.189576 +0000 +0000  0  NULL  NULL
        	            	857952512441253889  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:32:08.2597 +0000 +0000    NULL                                    NULL                                    2023-04-19 09:32:11.794378 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:08.2597 +0000 +0000    0  NULL  NULL
        	            	857952512680198145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-19 09:32:08.332627 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:08.332627 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:08.332627 +0000 +0000  0  NULL  NULL
        	            	857952512925040641  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-19 09:32:08.40733 +0000 +0000   NULL                                    NULL                                    2023-04-19 09:32:08.40733 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:08.40733 +0000 +0000   0  NULL  NULL
        	            	857952513187184641  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-19 09:32:08.487327 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:08.487327 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:08.487327 +0000 +0000  0  NULL  NULL
        	            	857952513424588801  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-19 09:32:08.559782 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:08.559782 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:08.559782 +0000 +0000  0  NULL  NULL
        	            	857952513568800769  AUTO CREATE STATS                Table statistics refresh for system.public.database_role_settings                       CREATE STATISTICS __auto__ FROM [44] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-19 09:32:08.603777 +0000 +0000  2023-04-19 09:32:08.752797 +0000 +0000  2023-04-19 09:32:09.487637 +0000 +0000  2023-04-19 09:32:09.487697 +0000 +0000  1  NULL    1     3994245837445585094  2023-04-19 09:32:08.752813 +0000 +0000  2023-04-20 09:32:08.752813 +0000 +0000  1  NULL  NULL
        	            	857952513652326401  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-19 09:32:08.629287 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:08.629287 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:08.629287 +0000 +0000  0  NULL  NULL
        	            	857952513905655809  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-19 09:32:08.706587 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:08.706587 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:08.706587 +0000 +0000  0  NULL  NULL
        	            	857952514146598913  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:32:08.78012 +0000 +0000   NULL                                    NULL                                    2023-04-19 09:32:11.365966 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:08.78012 +0000 +0000   0  NULL  NULL
        	            	857952514378891265  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:32:08.851023 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:11.449545 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:08.851023 +0000 +0000  0  NULL  NULL
        	            	857952514614362113  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:32:08.922873 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:11.53562 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:08.922873 +0000 +0000  0  NULL  NULL
        	            	857952514852945921  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-19 09:32:08.995766 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:08.995766 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:08.995766 +0000 +0000  0  NULL  NULL
        	            	857952515089399809  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-19 09:32:09.067901 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:09.067901 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:09.067901 +0000 +0000  0  NULL  NULL
        	            	857952515345317889  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-19 09:32:09.145938 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:09.145938 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:09.145938 +0000 +0000  0  NULL  NULL
        	            	857952515584983041  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-19 09:32:09.219083 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:09.219083 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:09.219083 +0000 +0000  0  NULL  NULL
        	            	857952515822518273  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-19 09:32:09.291572 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:09.291572 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:09.291572 +0000 +0000  0  NULL  NULL
        	            	857952516080467969  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-19 09:32:09.370302 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:09.370302 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:09.370302 +0000 +0000  0  NULL  NULL
        	            	857952516350476289  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-19 09:32:09.452699 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:09.452699 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:09.452699 +0000 +0000  0  NULL  NULL
        	            	857952516600954881  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-19 09:32:09.529125 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:09.529125 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:09.529125 +0000 +0000  0  NULL  NULL
        	            	857952516844224513  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-19 09:32:09.603377 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:09.603377 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:09.603377 +0000 +0000  0  NULL  NULL
        	            	857952517088739329  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:32:09.677993 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:09.677993 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:09.677993 +0000 +0000  0  NULL  NULL
        	            	857952517326503937  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:32:09.75055 +0000 +0000   NULL                                    NULL                                    2023-04-19 09:32:09.75055 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:09.75055 +0000 +0000   0  NULL  NULL
        	            	857952517574492161  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:32:09.826239 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:32:09.826239 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:32:09.826239 +0000 +0000  0  NULL  NULL
        	            	857952517700124673  AUTO CREATE STATS                Table statistics refresh for system.public.ui                                           CREATE STATISTICS __auto__ FROM [14] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-19 09:32:09.864555 +0000 +0000  2023-04-19 09:32:10.036002 +0000 +0000  NULL                                    2023-04-19 09:32:09.959742 +0000 +0000  0  NULL    1     3496245897193380098  2023-04-19 09:32:10.036022 +0000 +0000  2023-04-20 09:32:10.036022 +0000 +0000  1  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (10.46s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 2f86475f2e3d713b4c8c976c34dba99acce7c4c7:

        	            	857955423373426689  BACKUP                           fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-19 09:46:56.605703 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:56.605703 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:56.605703 +0000 +0000  0  NULL  NULL
        	            	857955423490736129  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-19 09:46:56.641487 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:56.641487 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:56.641487 +0000 +0000  0  NULL  NULL
        	            	857955423612370945  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-19 09:46:56.678608 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:56.678608 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:56.678608 +0000 +0000  0  NULL  NULL
        	            	857955423730532353  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-19 09:46:56.714671 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:56.714671 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:56.714671 +0000 +0000  0  NULL  NULL
        	            	857955423851151361  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-19 09:46:56.751486 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:56.751486 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:56.751486 +0000 +0000  0  NULL  NULL
        	            	857955423972229121  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-19 09:46:56.788439 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:56.788439 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:56.788439 +0000 +0000  0  NULL  NULL
        	            	857955424094420993  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-19 09:46:56.825718 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:56.825718 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:56.825718 +0000 +0000  0  NULL  NULL
        	            	857955424221790209  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-19 09:46:56.864601 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:56.864601 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:56.864601 +0000 +0000  0  NULL  NULL
        	            	857955424351092737  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-19 09:46:56.904056 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:56.904056 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:56.904056 +0000 +0000  0  NULL  NULL
        	            	857955424479150081  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-19 09:46:56.943138 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:56.943138 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:56.943138 +0000 +0000  0  NULL  NULL
        	            	857955424611467265  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-19 09:46:56.983512 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:56.983512 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:56.983512 +0000 +0000  0  NULL  NULL
        	            	857955424737198081  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-19 09:46:57.021897 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.021897 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.021897 +0000 +0000  0  NULL  NULL
        	            	857955424870006785  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-19 09:46:57.062416 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.062416 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.062416 +0000 +0000  0  NULL  NULL
        	            	857955425024114689  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-19 09:46:57.109438 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.109438 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.109438 +0000 +0000  0  NULL  NULL
        	            	857955425144700929  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-19 09:46:57.14624 +0000 +0000   NULL                                    NULL                                    2023-04-19 09:46:57.14624 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.14624 +0000 +0000   0  NULL  NULL
        	            	857955425269612545  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-19 09:46:57.184357 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.184357 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.184357 +0000 +0000  0  NULL  NULL
        	            	857955425391968257  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-19 09:46:57.221703 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.221703 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.221703 +0000 +0000  0  NULL  NULL
        	            	857955425512423425  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-19 09:46:57.258467 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.258467 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.258467 +0000 +0000  0  NULL  NULL
        	            	857955425638055937  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-19 09:46:57.296807 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.296807 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.296807 +0000 +0000  0  NULL  NULL
        	            	857955425757921281  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:46:57.333379 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.333379 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.333379 +0000 +0000  0  NULL  NULL
        	            	857955425880080385  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:46:57.370667 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.370667 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.370667 +0000 +0000  0  NULL  NULL
        	            	857955426009284609  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:46:57.410094 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.410094 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.410094 +0000 +0000  0  NULL  NULL
        	            	857955426139963393  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-19 09:46:57.44997 +0000 +0000   NULL                                    NULL                                    2023-04-19 09:46:57.44997 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.44997 +0000 +0000   0  NULL  NULL
        	            	857955426272772097  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-19 09:46:57.490512 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.490512 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.490512 +0000 +0000  0  NULL  NULL
        	            	857955426401583105  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-19 09:46:57.529819 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.529819 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.529819 +0000 +0000  0  NULL  NULL
        	            	857955426538422273  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-19 09:46:57.571578 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.571578 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.571578 +0000 +0000  0  NULL  NULL
        	            	857955426664513537  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-19 09:46:57.610063 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.610063 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.610063 +0000 +0000  0  NULL  NULL
        	            	857955426782085121  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-19 09:46:57.645935 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.645935 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.645935 +0000 +0000  0  NULL  NULL
        	            	857955426898051073  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-19 09:46:57.681325 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.681325 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.681325 +0000 +0000  0  NULL  NULL
        	            	857955427019849729  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-19 09:46:57.718492 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.718492 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.718492 +0000 +0000  0  NULL  NULL
        	            	857955427136634881  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-19 09:46:57.754136 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.754136 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.754136 +0000 +0000  0  NULL  NULL
        	            	857955427276423169  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-19 09:46:57.796796 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.796796 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.796796 +0000 +0000  0  NULL  NULL
        	            	857955427403137025  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-19 09:46:57.835458 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.835458 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.835458 +0000 +0000  0  NULL  NULL
        	            	857955427534766081  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-19 09:46:57.875659 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.875659 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.875659 +0000 +0000  0  NULL  NULL
        	            	857955427660562433  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-19 09:46:57.914022 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.914022 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.914022 +0000 +0000  0  NULL  NULL
        	            	857955427788914689  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-19 09:46:57.953192 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.953192 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.953192 +0000 +0000  0  NULL  NULL
        	            	857955427919134721  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-19 09:46:57.992938 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:57.992938 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:57.992938 +0000 +0000  0  NULL  NULL
        	            	857955428041228289  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-19 09:46:58.030202 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:58.030202 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:58.030202 +0000 +0000  0  NULL  NULL
        	            	857955428291379201  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-19 09:46:58.106597 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:58.106597 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:58.106597 +0000 +0000  0  NULL  NULL
        	            	857955428434214913  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-19 09:46:58.150123 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:58.150123 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:58.150123 +0000 +0000  0  NULL  NULL
        	            	857955428564434945  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-19 09:46:58.189874 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:58.189874 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:58.189874 +0000 +0000  0  NULL  NULL
        	            	857955428700323841  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-19 09:46:58.231339 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:58.231339 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:58.231339 +0000 +0000  0  NULL  NULL
        	            	857955428834836481  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-19 09:46:58.272384 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:58.272384 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:58.272384 +0000 +0000  0  NULL  NULL
        	            	857955428972494849  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-19 09:46:58.314401 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:58.314401 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:58.314401 +0000 +0000  0  NULL  NULL
        	            	857955429135056897  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-19 09:46:58.364014 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:58.364014 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:58.364014 +0000 +0000  0  NULL  NULL
        	            	857955429280940033  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-19 09:46:58.408519 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:58.408519 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:58.408519 +0000 +0000  0  NULL  NULL
        	            	857955429425545217  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:46:58.452656 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:58.452656 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:58.452656 +0000 +0000  0  NULL  NULL
        	            	857955429730254849  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:46:58.545679 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:58.545679 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:58.545679 +0000 +0000  0  NULL  NULL
        	            	857955429869813761  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 09:46:58.588232 +0000 +0000  NULL                                    NULL                                    2023-04-19 09:46:58.588232 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 09:46:58.588232 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (7.77s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ ad6ce866ea3b5c5bb47ba9a0ac19b721a0c98add:

        	            	858238931616497665  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 09:48:56.455284 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:56.455284 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:56.455284 +0000 +0000  0  NULL  NULL
        	            	858238931784466433  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 09:48:56.506528 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:56.506528 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:56.506528 +0000 +0000  0  NULL  NULL
        	            	858238931937787905  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 09:48:56.553316 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:56.553316 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:56.553316 +0000 +0000  0  NULL  NULL
        	            	858238932091174913  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 09:48:56.600148 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:56.600148 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:56.600148 +0000 +0000  0  NULL  NULL
        	            	858238932226703361  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 09:48:56.64149 +0000 +0000   NULL                                    NULL                                    2023-04-20 09:48:56.64149 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:56.64149 +0000 +0000   0  NULL  NULL
        	            	858238932362592257  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 09:48:56.682952 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:56.682952 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:56.682952 +0000 +0000  0  NULL  NULL
        	            	858238932490911745  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-20 09:48:56.722117 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:56.722117 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:56.722117 +0000 +0000  0  NULL  NULL
        	            	858238932624670721  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-20 09:48:56.76294 +0000 +0000   NULL                                    NULL                                    2023-04-20 09:48:56.76294 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:56.76294 +0000 +0000   0  NULL  NULL
        	            	858238932763213825  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-20 09:48:56.805208 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:56.805208 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:56.805208 +0000 +0000  0  NULL  NULL
        	            	858238932905656321  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 09:48:56.848686 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:56.848686 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:56.848686 +0000 +0000  0  NULL  NULL
        	            	858238933037285377  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 09:48:56.888854 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:56.888854 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:56.888854 +0000 +0000  0  NULL  NULL
        	            	858238933194637313  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 09:48:56.936876 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:56.936876 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:56.936876 +0000 +0000  0  NULL  NULL
        	            	858238933327446017  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 09:48:56.977405 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:56.977405 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:56.977405 +0000 +0000  0  NULL  NULL
        	            	858238933491810305  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 09:48:57.027572 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:57.027572 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.027572 +0000 +0000  0  NULL  NULL
        	            	858238933640413185  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 09:48:57.072915 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:57.072915 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.072915 +0000 +0000  0  NULL  NULL
        	            	858238933804580865  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 09:48:57.123021 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:57.123021 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.123021 +0000 +0000  0  NULL  NULL
        	            	858238933967241217  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 09:48:57.172673 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:57.172673 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.172673 +0000 +0000  0  NULL  NULL
        	            	858238934232563713  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 09:48:57.253636 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:57.253636 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.253636 +0000 +0000  0  NULL  NULL
        	            	858238934367371265  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:48:57.294763 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:57.294763 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.294763 +0000 +0000  0  NULL  NULL
        	            	858238934495428609  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:48:57.333843 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:57.333843 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.333843 +0000 +0000  0  NULL  NULL
        	            	858238934625878017  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:48:57.373664 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:57.373664 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.373664 +0000 +0000  0  NULL  NULL
        	            	858238934758260737  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:48:57.414062 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:49:01.044721 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.414062 +0000 +0000  0  NULL  NULL
        	            	858238934899130369  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:48:57.457056 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:49:01.110839 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.457056 +0000 +0000  0  NULL  NULL
        	            	858238935062085633  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:48:57.506781 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:49:01.176411 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.506781 +0000 +0000  0  NULL  NULL
        	            	858238935222091777  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:48:57.555603 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:49:01.889318 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.555603 +0000 +0000  0  NULL  NULL
        	            	858238935341039617  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:48:57.59191 +0000 +0000   NULL                                    NULL                                    2023-04-20 09:49:01.955668 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.59191 +0000 +0000   0  NULL  NULL
        	            	858238935465558017  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:48:57.629906 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:49:02.033515 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.629906 +0000 +0000  0  NULL  NULL
        	            	858238935590010881  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 09:48:57.667887 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:57.667887 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.667887 +0000 +0000  0  NULL  NULL
        	            	858238935715840001  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 09:48:57.706285 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:57.706285 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.706285 +0000 +0000  0  NULL  NULL
        	            	858238935842390017  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 09:48:57.744921 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:57.744921 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.744921 +0000 +0000  0  NULL  NULL
        	            	858238935977361409  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 09:48:57.78609 +0000 +0000   NULL                                    NULL                                    2023-04-20 09:48:57.78609 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.78609 +0000 +0000   0  NULL  NULL
        	            	858238936131829761  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 09:48:57.833237 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:57.833237 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.833237 +0000 +0000  0  NULL  NULL
        	            	858238936282562561  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 09:48:57.879231 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:57.879231 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.879231 +0000 +0000  0  NULL  NULL
        	            	858238936418648065  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:48:57.920769 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:49:01.678803 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.920769 +0000 +0000  0  NULL  NULL
        	            	858238936545624065  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:48:57.959527 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:49:01.743526 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:57.959527 +0000 +0000  0  NULL  NULL
        	            	858238936687607809  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:48:58.002848 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:49:01.810295 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:58.002848 +0000 +0000  0  NULL  NULL
        	            	858238936823136257  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 09:48:58.044236 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:58.044236 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:58.044236 +0000 +0000  0  NULL  NULL
        	            	858238936955289601  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 09:48:58.084551 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:58.084551 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:58.084551 +0000 +0000  0  NULL  NULL
        	            	858238937094291457  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 09:48:58.126965 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:58.126965 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:58.126965 +0000 +0000  0  NULL  NULL
        	            	858238937258131457  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 09:48:58.176963 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:58.176963 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:58.176963 +0000 +0000  0  NULL  NULL
        	            	858238937391726593  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 09:48:58.217724 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:58.217724 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:58.217724 +0000 +0000  0  NULL  NULL
        	            	858238937539674113  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 09:48:58.262875 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:58.262875 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:58.262875 +0000 +0000  0  NULL  NULL
        	            	858238937668845569  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 09:48:58.30229 +0000 +0000   NULL                                    NULL                                    2023-04-20 09:48:58.30229 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:58.30229 +0000 +0000   0  NULL  NULL
        	            	858238937804603393  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 09:48:58.343734 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:58.343734 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:58.343734 +0000 +0000  0  NULL  NULL
        	            	858238937932038145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 09:48:58.382614 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:58.382614 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:58.382614 +0000 +0000  0  NULL  NULL
        	            	858238938062782465  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:48:58.422516 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:58.422516 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:58.422516 +0000 +0000  0  NULL  NULL
        	            	858238938192904193  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:48:58.462224 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:58.462224 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:58.462224 +0000 +0000  0  NULL  NULL
        	            	858238938321092609  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:48:58.501344 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:48:58.501344 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:48:58.501344 +0000 +0000  0  NULL  NULL
        	            	858238938416414721  AUTO CREATE STATS                Table statistics refresh for system.public.statement_diagnostics                        CREATE STATISTICS __auto__ FROM [36] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-20 09:48:58.53041 +0000 +0000   2023-04-20 09:48:58.637773 +0000 +0000  NULL                                    2023-04-20 09:48:58.599011 +0000 +0000  0  NULL    1     2962495186579654658  2023-04-20 09:48:58.637778 +0000 +0000  2023-04-21 09:48:58.637778 +0000 +0000  1  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (9.25s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ ad6ce866ea3b5c5bb47ba9a0ac19b721a0c98add:

        	            	858238441584132097  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 09:46:26.909246 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:26.909246 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:26.909246 +0000 +0000  0  NULL  NULL
        	            	858238441805905921  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 09:46:26.976924 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:26.976924 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:26.976924 +0000 +0000  0  NULL  NULL
        	            	858238442057007105  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 09:46:27.053557 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:27.053557 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:27.053557 +0000 +0000  0  NULL  NULL
        	            	858238442355195905  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 09:46:27.14456 +0000 +0000   NULL                                    NULL                                    2023-04-20 09:46:27.14456 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:27.14456 +0000 +0000   0  NULL  NULL
        	            	858238442579820545  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 09:46:27.213132 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:27.213132 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:27.213132 +0000 +0000  0  NULL  NULL
        	            	858238442812407809  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 09:46:27.284089 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:27.284089 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:27.284089 +0000 +0000  0  NULL  NULL
        	            	858238443026513921  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-20 09:46:27.349431 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:27.349431 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:27.349431 +0000 +0000  0  NULL  NULL
        	            	858238443245797377  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-20 09:46:27.416349 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:27.416349 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:27.416349 +0000 +0000  0  NULL  NULL
        	            	858238443498897409  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-20 09:46:27.493583 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:27.493583 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:27.493583 +0000 +0000  0  NULL  NULL
        	            	858238443725225985  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 09:46:27.562653 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:27.562653 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:27.562653 +0000 +0000  0  NULL  NULL
        	            	858238443954143233  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 09:46:27.632515 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:27.632515 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:27.632515 +0000 +0000  0  NULL  NULL
        	            	858238444169297921  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 09:46:27.698182 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:27.698182 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:27.698182 +0000 +0000  0  NULL  NULL
        	            	858238444390383617  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 09:46:27.765643 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:27.765643 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:27.765643 +0000 +0000  0  NULL  NULL
        	            	858238444612452353  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 09:46:27.83342 +0000 +0000   NULL                                    NULL                                    2023-04-20 09:46:27.83342 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:27.83342 +0000 +0000   0  NULL  NULL
        	            	858238444829310977  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 09:46:27.899593 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:27.899593 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:27.899593 +0000 +0000  0  NULL  NULL
        	            	858238445085949953  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 09:46:27.977911 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:27.977911 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:27.977911 +0000 +0000  0  NULL  NULL
        	            	858238445348061185  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 09:46:28.057911 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:28.057911 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:28.057911 +0000 +0000  0  NULL  NULL
        	            	858238445592674305  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 09:46:28.132552 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:28.132552 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:28.132552 +0000 +0000  0  NULL  NULL
        	            	858238445935525889  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:46:28.237188 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:28.237188 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:28.237188 +0000 +0000  0  NULL  NULL
        	            	858238446154481665  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:46:28.304004 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:28.304004 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:28.304004 +0000 +0000  0  NULL  NULL
        	            	858238446393851905  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:46:28.37706 +0000 +0000   NULL                                    NULL                                    2023-04-20 09:46:28.37706 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:28.37706 +0000 +0000   0  NULL  NULL
        	            	858238446617690113  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:46:28.445369 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:36.143367 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:28.445369 +0000 +0000  0  NULL  NULL
        	            	858238446828322817  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:46:28.509646 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:36.225524 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:28.509646 +0000 +0000  0  NULL  NULL
        	            	858238447056715777  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:46:28.579343 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:36.303693 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:28.579343 +0000 +0000  0  NULL  NULL
        	            	858238447284027393  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:46:28.648713 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:37.523071 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:28.648713 +0000 +0000  0  NULL  NULL
        	            	858238447534538753  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:46:28.725167 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:37.602538 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:28.725167 +0000 +0000  0  NULL  NULL
        	            	858238447751299073  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:46:28.791318 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:37.682541 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:28.791318 +0000 +0000  0  NULL  NULL
        	            	858238447965765633  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 09:46:28.856777 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:28.856777 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:28.856777 +0000 +0000  0  NULL  NULL
        	            	858238448189702145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 09:46:28.925103 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:28.925103 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:28.925103 +0000 +0000  0  NULL  NULL
        	            	858238448469737473  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 09:46:29.010571 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:29.010571 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:29.010571 +0000 +0000  0  NULL  NULL
        	            	858238448712613889  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 09:46:29.084685 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:29.084685 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:29.084685 +0000 +0000  0  NULL  NULL
        	            	858238448937664513  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 09:46:29.15337 +0000 +0000   NULL                                    NULL                                    2023-04-20 09:46:29.15337 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:29.15337 +0000 +0000   0  NULL  NULL
        	            	858238449160486913  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 09:46:29.221365 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:29.221365 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:29.221365 +0000 +0000  0  NULL  NULL
        	            	858238449385177089  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:46:29.289944 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:37.273178 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:29.289944 +0000 +0000  0  NULL  NULL
        	            	858238449647026177  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:46:29.369851 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:37.354603 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:29.369851 +0000 +0000  0  NULL  NULL
        	            	858238449868013569  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:46:29.437286 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:37.435847 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:29.437286 +0000 +0000  0  NULL  NULL
        	            	858238450092441601  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 09:46:29.505778 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:29.505778 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:29.505778 +0000 +0000  0  NULL  NULL
        	            	858238450318901249  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 09:46:29.574899 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:29.574899 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:29.574899 +0000 +0000  0  NULL  NULL
        	            	858238450558205953  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 09:46:29.647918 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:29.647918 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:29.647918 +0000 +0000  0  NULL  NULL
        	            	858238450786304001  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 09:46:29.717531 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:29.717531 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:29.717531 +0000 +0000  0  NULL  NULL
        	            	858238451010371585  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 09:46:29.785912 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:29.785912 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:29.785912 +0000 +0000  0  NULL  NULL
        	            	858238451247120385  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 09:46:29.858162 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:29.858162 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:29.858162 +0000 +0000  0  NULL  NULL
        	            	858238451514834945  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 09:46:29.93987 +0000 +0000   NULL                                    NULL                                    2023-04-20 09:46:29.93987 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:29.93987 +0000 +0000   0  NULL  NULL
        	            	858238451754598401  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 09:46:30.01303 +0000 +0000   NULL                                    NULL                                    2023-04-20 09:46:30.01303 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:30.01303 +0000 +0000   0  NULL  NULL
        	            	858238451992625153  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 09:46:30.085684 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:30.085684 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:30.085684 +0000 +0000  0  NULL  NULL
        	            	858238452277280769  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:46:30.172534 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:30.172534 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:30.172534 +0000 +0000  0  NULL  NULL
        	            	858238452616069121  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:46:30.275936 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:30.275936 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:30.275936 +0000 +0000  0  NULL  NULL
        	            	858238452840497153  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 09:46:30.344416 +0000 +0000  NULL                                    NULL                                    2023-04-20 09:46:30.344416 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 09:46:30.344416 +0000 +0000  0  NULL  NULL
        	            	858238456815550465  AUTO CREATE STATS                Table statistics refresh for system.public.comments                                     CREATE STATISTICS __auto__ FROM [24] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-20 09:46:31.557493 +0000 +0000  2023-04-20 09:46:31.727887 +0000 +0000  NULL                                    2023-04-20 09:46:31.676414 +0000 +0000  0  NULL    1     1963390707683710113  2023-04-20 09:46:31.727903 +0000 +0000  2023-04-21 09:46:31.727903 +0000 +0000  1  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (18.02s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ c54981d0acec7762c85cb78a868ac8af193c9bdc:

        	            	858536847295414273  BACKUP                           fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-21 11:04:13.10532 +0000 +0000   NULL                                    NULL                                    2023-04-21 11:04:13.10532 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.10532 +0000 +0000   0  NULL  NULL
        	            	858536847418851329  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-21 11:04:13.142996 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.142996 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.142996 +0000 +0000  0  NULL  NULL
        	            	858536847553658881  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-21 11:04:13.184144 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.184144 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.184144 +0000 +0000  0  NULL  NULL
        	            	858536847700295681  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-21 11:04:13.228892 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.228892 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.228892 +0000 +0000  0  NULL  NULL
        	            	858536847843655681  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 11:04:13.272637 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.272637 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.272637 +0000 +0000  0  NULL  NULL
        	            	858536847978627073  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 11:04:13.313829 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.313829 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.313829 +0000 +0000  0  NULL  NULL
        	            	858536848115335169  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 11:04:13.355539 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.355539 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.355539 +0000 +0000  0  NULL  NULL
        	            	858536848251420673  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-21 11:04:13.397093 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.397093 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.397093 +0000 +0000  0  NULL  NULL
        	            	858536848426598401  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-21 11:04:13.450531 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.450531 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.450531 +0000 +0000  0  NULL  NULL
        	            	858536848570974209  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-21 11:04:13.494599 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.494599 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.494599 +0000 +0000  0  NULL  NULL
        	            	858536848705978369  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 11:04:13.535793 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.535793 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.535793 +0000 +0000  0  NULL  NULL
        	            	858536848835051521  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 11:04:13.575183 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.575183 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.575183 +0000 +0000  0  NULL  NULL
        	            	858536848966975489  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 11:04:13.615448 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.615448 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.615448 +0000 +0000  0  NULL  NULL
        	            	858536849100111873  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 11:04:13.656078 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.656078 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.656078 +0000 +0000  0  NULL  NULL
        	            	858536849247797249  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 11:04:13.701143 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.701143 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.701143 +0000 +0000  0  NULL  NULL
        	            	858536849382539265  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 11:04:13.742261 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.742261 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.742261 +0000 +0000  0  NULL  NULL
        	            	858536849515675649  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 11:04:13.782895 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.782895 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.782895 +0000 +0000  0  NULL  NULL
        	            	858536849666703361  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 11:04:13.828995 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.828995 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.828995 +0000 +0000  0  NULL  NULL
        	            	858536849804689409  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 11:04:13.871091 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.871091 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.871091 +0000 +0000  0  NULL  NULL
        	            	858536849932779521  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:04:13.910187 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.910187 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.910187 +0000 +0000  0  NULL  NULL
        	            	858536850065260545  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:04:13.950619 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.950619 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.950619 +0000 +0000  0  NULL  NULL
        	            	858536850197774337  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:04:13.991055 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:13.991055 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:13.991055 +0000 +0000  0  NULL  NULL
        	            	858536850330288129  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:04:14.031491 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:17.540022 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.031491 +0000 +0000  0  NULL  NULL
        	            	858536850462998529  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:04:14.071998 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:17.616084 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.071998 +0000 +0000  0  NULL  NULL
        	            	858536850593185793  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:04:14.111727 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:17.68528 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.111727 +0000 +0000  0  NULL  NULL
        	            	858536850723471361  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:04:14.151494 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:18.409049 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.151494 +0000 +0000  0  NULL  NULL
        	            	858536850851168257  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:04:14.190459 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:18.478262 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.190459 +0000 +0000  0  NULL  NULL
        	            	858536850976735233  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:04:14.228778 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:18.545932 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.228778 +0000 +0000  0  NULL  NULL
        	            	858536851098828801  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-21 11:04:14.266032 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:14.266032 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.266032 +0000 +0000  0  NULL  NULL
        	            	858536851231473665  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-21 11:04:14.306512 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:14.306512 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.306512 +0000 +0000  0  NULL  NULL
        	            	858536851396886529  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-21 11:04:14.357005 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:14.357005 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.357005 +0000 +0000  0  NULL  NULL
        	            	858536851549224961  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 11:04:14.403486 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:14.403486 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.403486 +0000 +0000  0  NULL  NULL
        	            	858536851689701377  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 11:04:14.44636 +0000 +0000   NULL                                    NULL                                    2023-04-21 11:04:14.44636 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.44636 +0000 +0000   0  NULL  NULL
        	            	858536851823919105  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 11:04:14.487328 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:14.487328 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.487328 +0000 +0000  0  NULL  NULL
        	            	858536851986022401  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:04:14.536799 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:18.195428 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.536799 +0000 +0000  0  NULL  NULL
        	            	858536852117651457  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:04:14.576963 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:18.262037 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.576963 +0000 +0000  0  NULL  NULL
        	            	858536852244135937  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:04:14.615558 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:18.330119 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.615558 +0000 +0000  0  NULL  NULL
        	            	858536852370817025  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 11:04:14.65421 +0000 +0000   NULL                                    NULL                                    2023-04-21 11:04:14.65421 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.65421 +0000 +0000   0  NULL  NULL
        	            	858536852505001985  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 11:04:14.695167 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:14.695167 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.695167 +0000 +0000  0  NULL  NULL
        	            	858536852638040065  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 11:04:14.735768 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:14.735768 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.735768 +0000 +0000  0  NULL  NULL
        	            	858536852768587777  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 11:04:14.775611 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:14.775611 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.775611 +0000 +0000  0  NULL  NULL
        	            	858536852903755777  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 11:04:14.816854 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:14.816854 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.816854 +0000 +0000  0  NULL  NULL
        	            	858536853038039041  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 11:04:14.857839 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:14.857839 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.857839 +0000 +0000  0  NULL  NULL
        	            	858536853169569793  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 11:04:14.897972 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:14.897972 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.897972 +0000 +0000  0  NULL  NULL
        	            	858536853306376193  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 11:04:14.939726 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:14.939726 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.939726 +0000 +0000  0  NULL  NULL
        	            	858536853434597377  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 11:04:14.978856 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:14.978856 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:14.978856 +0000 +0000  0  NULL  NULL
        	            	858536853568815105  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:04:15.019829 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:15.019829 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:15.019829 +0000 +0000  0  NULL  NULL
        	            	858536853709520897  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:04:15.062761 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:15.062761 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:15.062761 +0000 +0000  0  NULL  NULL
        	            	858536853873229825  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:04:15.112719 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:04:15.112719 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:04:15.112719 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (9.99s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ c54981d0acec7762c85cb78a868ac8af193c9bdc:

        	            	858544990507073537  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-21 11:45:38.216299 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:38.216299 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:38.216299 +0000 +0000  0  NULL  NULL
        	            	858544990739791873  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 11:45:38.287317 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:38.287317 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:38.287317 +0000 +0000  0  NULL  NULL
        	            	858544990967562241  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 11:45:38.356833 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:38.356833 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:38.356833 +0000 +0000  0  NULL  NULL
        	            	858544991234686977  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 11:45:38.438346 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:38.438346 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:38.438346 +0000 +0000  0  NULL  NULL
        	            	858544991460950017  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-21 11:45:38.507393 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:38.507393 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:38.507393 +0000 +0000  0  NULL  NULL
        	            	858544991709462529  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-21 11:45:38.583234 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:38.583234 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:38.583234 +0000 +0000  0  NULL  NULL
        	            	858544991809667073  AUTO CREATE STATS                Table statistics refresh for system.public.join_tokens                                  CREATE STATISTICS __auto__ FROM [41] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-21 11:45:38.6138 +0000 +0000    2023-04-21 11:45:38.765138 +0000 +0000  2023-04-21 11:45:39.711649 +0000 +0000  2023-04-21 11:45:39.711699 +0000 +0000  1  NULL    1     6574549783095085951  2023-04-21 11:45:38.765293 +0000 +0000  2023-04-22 11:45:38.765293 +0000 +0000  1  NULL  NULL
        	            	858544991938805761  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-21 11:45:38.653221 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:38.653221 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:38.653221 +0000 +0000  0  NULL  NULL
        	            	858544992196263937  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 11:45:38.733244 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:38.733244 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:38.733244 +0000 +0000  0  NULL  NULL
        	            	858544992439566337  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 11:45:38.806042 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:38.806042 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:38.806042 +0000 +0000  0  NULL  NULL
        	            	858544992672284673  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 11:45:38.877062 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:38.877062 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:38.877062 +0000 +0000  0  NULL  NULL
        	            	858544992892092417  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 11:45:38.944142 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:38.944142 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:38.944142 +0000 +0000  0  NULL  NULL
        	            	858544993117208577  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 11:45:39.012846 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:39.012846 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:39.012846 +0000 +0000  0  NULL  NULL
        	            	858544993409957889  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 11:45:39.102187 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:39.102187 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:39.102187 +0000 +0000  0  NULL  NULL
        	            	858544993614692353  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 11:45:39.164675 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:39.164675 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:39.164675 +0000 +0000  0  NULL  NULL
        	            	858544993841381377  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 11:45:39.233855 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:39.233855 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:39.233855 +0000 +0000  0  NULL  NULL
        	            	858544994069610497  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 11:45:39.303497 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:39.303497 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:39.303497 +0000 +0000  0  NULL  NULL
        	            	858544994289909761  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:45:39.370731 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:39.370731 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:39.370731 +0000 +0000  0  NULL  NULL
        	            	858544994526101505  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:45:39.442803 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:39.442803 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:39.442803 +0000 +0000  0  NULL  NULL
        	            	858544994754920449  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:45:39.512644 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:39.512644 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:39.512644 +0000 +0000  0  NULL  NULL
        	            	858544994986000385  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-21 11:45:39.583155 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:39.583155 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:39.583155 +0000 +0000  0  NULL  NULL
        	            	858544995207905281  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-21 11:45:39.650878 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:39.650878 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:39.650878 +0000 +0000  0  NULL  NULL
        	            	858544995439738881  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-21 11:45:39.721627 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:39.721627 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:39.721627 +0000 +0000  0  NULL  NULL
        	            	858544995715579905  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-21 11:45:39.80581 +0000 +0000   NULL                                    NULL                                    2023-04-21 11:45:39.80581 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:39.80581 +0000 +0000   0  NULL  NULL
        	            	858544995969957889  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-21 11:45:39.883438 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:39.883438 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:39.883438 +0000 +0000  0  NULL  NULL
        	            	858544996205264897  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-21 11:45:39.955251 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:39.955251 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:39.955251 +0000 +0000  0  NULL  NULL
        	            	858544996440080385  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-21 11:45:40.026906 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:40.026906 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:40.026906 +0000 +0000  0  NULL  NULL
        	            	858544996650352641  AUTO CREATE STATS                Table statistics refresh for system.public.privileges                                   CREATE STATISTICS __auto__ FROM [51] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-21 11:45:40.091063 +0000 +0000  2023-04-21 11:45:40.246913 +0000 +0000  2023-04-21 11:45:41.002024 +0000 +0000  2023-04-21 11:45:41.002082 +0000 +0000  1  NULL    1     3933898962186913230  2023-04-21 11:45:40.246927 +0000 +0000  2023-04-22 11:45:40.246927 +0000 +0000  1  NULL  NULL
        	            	858544996673880065  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-21 11:45:40.098256 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:40.098256 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:40.098256 +0000 +0000  0  NULL  NULL
        	            	858544996909350913  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-21 11:45:40.170122 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:40.170122 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:40.170122 +0000 +0000  0  NULL  NULL
        	            	858544997167333377  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 11:45:40.248845 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:40.248845 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:40.248845 +0000 +0000  0  NULL  NULL
        	            	858544997391695873  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 11:45:40.317319 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:40.317319 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:40.317319 +0000 +0000  0  NULL  NULL
        	            	858544997618024449  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 11:45:40.386384 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:40.386384 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:40.386384 +0000 +0000  0  NULL  NULL
        	            	858544997846351873  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-21 11:45:40.456073 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:40.456073 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:40.456073 +0000 +0000  0  NULL  NULL
        	            	858544998074941441  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-21 11:45:40.525824 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:40.525824 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:40.525824 +0000 +0000  0  NULL  NULL
        	            	858544998318047233  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-21 11:45:40.600021 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:40.600021 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:40.600021 +0000 +0000  0  NULL  NULL
        	            	858544998560628737  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 11:45:40.674038 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:40.674038 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:40.674038 +0000 +0000  0  NULL  NULL
        	            	858544998795673601  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 11:45:40.745784 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:40.745784 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:40.745784 +0000 +0000  0  NULL  NULL
        	            	858544999027539969  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 11:45:40.816533 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:40.816533 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:40.816533 +0000 +0000  0  NULL  NULL
        	            	858544999272808449  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 11:45:40.891385 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:40.891385 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:40.891385 +0000 +0000  0  NULL  NULL
        	            	858544999494123521  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 11:45:40.95893 +0000 +0000   NULL                                    NULL                                    2023-04-21 11:45:40.95893 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:40.95893 +0000 +0000   0  NULL  NULL
        	            	858544999728152577  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 11:45:41.030351 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:41.030351 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:41.030351 +0000 +0000  0  NULL  NULL
        	            	858544999957266433  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 11:45:41.100275 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:41.100275 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:41.100275 +0000 +0000  0  NULL  NULL
        	            	858545000178253825  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 11:45:41.167717 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:41.167717 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:41.167717 +0000 +0000  0  NULL  NULL
        	            	858545000419164161  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 11:45:41.241234 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:41.241234 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:41.241234 +0000 +0000  0  NULL  NULL
        	            	858545000651718657  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:45:41.312204 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:41.312204 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:41.312204 +0000 +0000  0  NULL  NULL
        	            	858545000895414273  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:45:41.386564 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:41.386564 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:41.386564 +0000 +0000  0  NULL  NULL
        	            	858545000930607105  AUTO CREATE STATS                Table statistics refresh for system.public.job_info                                     CREATE STATISTICS __auto__ FROM [53] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-21 11:45:41.397289 +0000 +0000  2023-04-21 11:45:41.553549 +0000 +0000  NULL                                    2023-04-21 11:45:41.499002 +0000 +0000  0  NULL    1     2995676759268653612  2023-04-21 11:45:41.553562 +0000 +0000  2023-04-22 11:45:41.553562 +0000 +0000  1  NULL  NULL
        	            	858545001133703169  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 11:45:41.459288 +0000 +0000  NULL                                    NULL                                    2023-04-21 11:45:41.459288 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 11:45:41.459288 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (8.81s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 01f129a59ab66166d6ef7189a8930f2b37cced74:

        	            	858804089394200577  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-22 09:43:28.9216 +0000 +0000    NULL                                    NULL                                    2023-04-22 09:43:28.9216 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:28.9216 +0000 +0000    0  NULL  NULL
        	            	858804089518358529  AUTO CREATE STATS                Table statistics refresh for system.public.privileges                                   CREATE STATISTICS __auto__ FROM [51] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-22 09:43:28.959473 +0000 +0000  2023-04-22 09:43:29.108769 +0000 +0000  2023-04-22 09:43:29.881286 +0000 +0000  2023-04-22 09:43:29.881344 +0000 +0000  1  NULL    1     550056915511533929   2023-04-22 09:43:29.108784 +0000 +0000  2023-04-23 09:43:29.108784 +0000 +0000  1  NULL  NULL
        	            	858804089619316737  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-22 09:43:28.990311 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:28.990311 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:28.990311 +0000 +0000  0  NULL  NULL
        	            	858804089850691585  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 09:43:29.060904 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:29.060904 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:29.060904 +0000 +0000  0  NULL  NULL
        	            	858804090095108097  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 09:43:29.135509 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:29.135509 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:29.135509 +0000 +0000  0  NULL  NULL
        	            	858804090312032257  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 09:43:29.201701 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:29.201701 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:29.201701 +0000 +0000  0  NULL  NULL
        	            	858804090543964161  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-22 09:43:29.272483 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:29.272483 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:29.272483 +0000 +0000  0  NULL  NULL
        	            	858804090790936577  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-22 09:43:29.347851 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:29.347851 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:29.347851 +0000 +0000  0  NULL  NULL
        	            	858804091031683073  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-22 09:43:29.421315 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:29.421315 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:29.421315 +0000 +0000  0  NULL  NULL
        	            	858804091264040961  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 09:43:29.492228 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:29.492228 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:29.492228 +0000 +0000  0  NULL  NULL
        	            	858804091501510657  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 09:43:29.5647 +0000 +0000    NULL                                    NULL                                    2023-04-22 09:43:29.5647 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:29.5647 +0000 +0000    0  NULL  NULL
        	            	858804091731804161  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 09:43:29.634968 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:29.634968 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:29.634968 +0000 +0000  0  NULL  NULL
        	            	858804091966750721  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 09:43:29.706686 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:29.706686 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:29.706686 +0000 +0000  0  NULL  NULL
        	            	858804092203859969  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 09:43:29.779039 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:29.779039 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:29.779039 +0000 +0000  0  NULL  NULL
        	            	858804092496281601  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 09:43:29.868279 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:29.868279 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:29.868279 +0000 +0000  0  NULL  NULL
        	            	858804092742303745  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 09:43:29.943356 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:29.943356 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:29.943356 +0000 +0000  0  NULL  NULL
        	            	858804092976496641  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 09:43:30.014832 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:30.014832 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:30.014832 +0000 +0000  0  NULL  NULL
        	            	858804093202169857  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 09:43:30.083744 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:30.083744 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:30.083744 +0000 +0000  0  NULL  NULL
        	            	858804093462380545  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:43:30.163103 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:30.163103 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:30.163103 +0000 +0000  0  NULL  NULL
        	            	858804093729341441  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:43:30.244578 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:30.244578 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:30.244578 +0000 +0000  0  NULL  NULL
        	            	858804093895737345  AUTO CREATE STATS                Table statistics refresh for system.public.statement_activity                           CREATE STATISTICS __auto__ FROM [60] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-22 09:43:30.295346 +0000 +0000  2023-04-22 09:43:30.45808 +0000 +0000   2023-04-22 09:43:32.406821 +0000 +0000  2023-04-22 09:43:32.406878 +0000 +0000  1  NULL    1     3301963376687376845  2023-04-22 09:43:30.458099 +0000 +0000  2023-04-23 09:43:30.458099 +0000 +0000  1  NULL  NULL
        	            	858804094011244545  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:43:30.330612 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:30.330612 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:30.330612 +0000 +0000  0  NULL  NULL
        	            	858804094300061697  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-22 09:43:30.418753 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:30.418753 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:30.418753 +0000 +0000  0  NULL  NULL
        	            	858804094536515585  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-22 09:43:30.49091 +0000 +0000   NULL                                    NULL                                    2023-04-22 09:43:30.49091 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:30.49091 +0000 +0000   0  NULL  NULL
        	            	858804094775427073  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-22 09:43:30.563834 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:30.563834 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:30.563834 +0000 +0000  0  NULL  NULL
        	            	858804095002214401  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-22 09:43:30.633025 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:30.633025 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:30.633025 +0000 +0000  0  NULL  NULL
        	            	858804095223267329  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-22 09:43:30.700486 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:30.700486 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:30.700486 +0000 +0000  0  NULL  NULL
        	            	858804095457656833  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-22 09:43:30.77203 +0000 +0000   NULL                                    NULL                                    2023-04-22 09:43:30.77203 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:30.77203 +0000 +0000   0  NULL  NULL
        	            	858804095670484993  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-22 09:43:30.836976 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:30.836976 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:30.836976 +0000 +0000  0  NULL  NULL
        	            	858804095900319745  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-22 09:43:30.907104 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:30.907104 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:30.907104 +0000 +0000  0  NULL  NULL
        	            	858804096137560065  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-22 09:43:30.979508 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:30.979508 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:30.979508 +0000 +0000  0  NULL  NULL
        	            	858804096376537089  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 09:43:31.052443 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:31.052443 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:31.052443 +0000 +0000  0  NULL  NULL
        	            	858804096630325249  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 09:43:31.129884 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:31.129884 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:31.129884 +0000 +0000  0  NULL  NULL
        	            	858804096864714753  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 09:43:31.201419 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:31.201419 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:31.201419 +0000 +0000  0  NULL  NULL
        	            	858804097127841793  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-22 09:43:31.281716 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:31.281716 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:31.281716 +0000 +0000  0  NULL  NULL
        	            	858804097366458369  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-22 09:43:31.354536 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:31.354536 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:31.354536 +0000 +0000  0  NULL  NULL
        	            	858804097603993601  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-22 09:43:31.427091 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:31.427091 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:31.427091 +0000 +0000  0  NULL  NULL
        	            	858804097842741249  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 09:43:31.49989 +0000 +0000   NULL                                    NULL                                    2023-04-22 09:43:31.49989 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:31.49989 +0000 +0000   0  NULL  NULL
        	            	858804098096562177  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 09:43:31.577348 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:31.577348 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:31.577348 +0000 +0000  0  NULL  NULL
        	            	858804098346254337  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 09:43:31.653556 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:31.653556 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:31.653556 +0000 +0000  0  NULL  NULL
        	            	858804098581561345  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 09:43:31.725353 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:31.725353 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:31.725353 +0000 +0000  0  NULL  NULL
        	            	858804098816540673  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 09:43:31.797063 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:31.797063 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:31.797063 +0000 +0000  0  NULL  NULL
        	            	858804099057188865  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 09:43:31.870515 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:31.870515 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:31.870515 +0000 +0000  0  NULL  NULL
        	            	858804099308748801  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 09:43:31.947276 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:31.947276 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:31.947276 +0000 +0000  0  NULL  NULL
        	            	858804099548676097  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 09:43:32.020501 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:32.020501 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:32.020501 +0000 +0000  0  NULL  NULL
        	            	858804099785162753  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 09:43:32.092679 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:32.092679 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:32.092679 +0000 +0000  0  NULL  NULL
        	            	858804100020305921  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:43:32.164426 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:32.164426 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:32.164426 +0000 +0000  0  NULL  NULL
        	            	858804100254269441  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:43:32.235834 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:32.235834 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:32.235834 +0000 +0000  0  NULL  NULL
        	            	858804100527128577  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:43:32.319105 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:43:32.319105 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:43:32.319105 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (9.19s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ 01f129a59ab66166d6ef7189a8930f2b37cced74:

        	            	858804437634154497  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-22 09:45:15.196006 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:15.196006 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.196006 +0000 +0000  0  NULL  NULL
        	            	858804437774434305  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-22 09:45:15.238851 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:15.238851 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.238851 +0000 +0000  0  NULL  NULL
        	            	858804437916254209  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-22 09:45:15.282101 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:15.282101 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.282101 +0000 +0000  0  NULL  NULL
        	            	858804438064463873  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 09:45:15.327328 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:15.327328 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.327328 +0000 +0000  0  NULL  NULL
        	            	858804438203531265  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 09:45:15.369771 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:15.369771 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.369771 +0000 +0000  0  NULL  NULL
        	            	858804438343942145  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 09:45:15.412614 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:15.412614 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.412614 +0000 +0000  0  NULL  NULL
        	            	858804438478192641  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-22 09:45:15.453585 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:15.453585 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.453585 +0000 +0000  0  NULL  NULL
        	            	858804438611591169  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-22 09:45:15.494301 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:15.494301 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.494301 +0000 +0000  0  NULL  NULL
        	            	858804438752296961  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-22 09:45:15.537236 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:15.537236 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.537236 +0000 +0000  0  NULL  NULL
        	            	858804438900015105  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 09:45:15.582321 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:15.582321 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.582321 +0000 +0000  0  NULL  NULL
        	            	858804439046455297  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 09:45:15.627012 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:15.627012 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.627012 +0000 +0000  0  NULL  NULL
        	            	858804439182901249  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 09:45:15.66864 +0000 +0000   NULL                                    NULL                                    2023-04-22 09:45:15.66864 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.66864 +0000 +0000   0  NULL  NULL
        	            	858804439317118977  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 09:45:15.709609 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:15.709609 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.709609 +0000 +0000  0  NULL  NULL
        	            	858804439450714113  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 09:45:15.750383 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:15.750383 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.750383 +0000 +0000  0  NULL  NULL
        	            	858804439588405249  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 09:45:15.792392 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:15.792392 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.792392 +0000 +0000  0  NULL  NULL
        	            	858804439763714049  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 09:45:15.84589 +0000 +0000   NULL                                    NULL                                    2023-04-22 09:45:15.84589 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.84589 +0000 +0000   0  NULL  NULL
        	            	858804439904911361  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 09:45:15.888985 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:15.888985 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.888985 +0000 +0000  0  NULL  NULL
        	            	858804440042373121  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 09:45:15.930949 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:15.930949 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.930949 +0000 +0000  0  NULL  NULL
        	            	858804440130781185  AUTO CREATE STATS                Table statistics refresh for system.public.span_stats_unique_keys                       CREATE STATISTICS __auto__ FROM [54] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-22 09:45:15.95789 +0000 +0000   2023-04-22 09:45:16.059026 +0000 +0000  2023-04-22 09:45:17.73919 +0000 +0000   2023-04-22 09:45:17.739286 +0000 +0000  1  NULL    1     6786277671026156943  2023-04-22 09:45:16.059032 +0000 +0000  2023-04-23 09:45:16.059032 +0000 +0000  1  NULL  NULL
        	            	858804440179998721  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:45:15.972937 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:15.972937 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:15.972937 +0000 +0000  0  NULL  NULL
        	            	858804440312414209  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:45:16.013354 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:16.013354 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.013354 +0000 +0000  0  NULL  NULL
        	            	858804440451973121  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:45:16.055936 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:16.055936 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.055936 +0000 +0000  0  NULL  NULL
        	            	858804440587894785  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:45:16.097413 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:20.000831 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.097413 +0000 +0000  0  NULL  NULL
        	            	858804440754421761  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:45:16.148234 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:20.070952 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.148234 +0000 +0000  0  NULL  NULL
        	            	858804440897191937  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:45:16.191809 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:20.144995 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.191809 +0000 +0000  0  NULL  NULL
        	            	858804441057263617  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:45:16.24066 +0000 +0000   NULL                                    NULL                                    2023-04-22 09:45:21.01145 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.24066 +0000 +0000   0  NULL  NULL
        	            	858804441192988673  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:45:16.282083 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:21.087906 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.282083 +0000 +0000  0  NULL  NULL
        	            	858804441333039105  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:45:16.324816 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:21.163738 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.324816 +0000 +0000  0  NULL  NULL
        	            	858804441485639681  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-22 09:45:16.371383 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:16.371383 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.371383 +0000 +0000  0  NULL  NULL
        	            	858804441633423361  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-22 09:45:16.416486 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:16.416486 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.416486 +0000 +0000  0  NULL  NULL
        	            	858804441809289217  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-22 09:45:16.470166 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:16.470166 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.470166 +0000 +0000  0  NULL  NULL
        	            	858804441963266049  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 09:45:16.51716 +0000 +0000   NULL                                    NULL                                    2023-04-22 09:45:16.51716 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.51716 +0000 +0000   0  NULL  NULL
        	            	858804442104725505  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 09:45:16.560321 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:16.560321 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.560321 +0000 +0000  0  NULL  NULL
        	            	858804442274070529  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 09:45:16.612002 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:16.612002 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.612002 +0000 +0000  0  NULL  NULL
        	            	858804442419134465  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:45:16.656277 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:20.75711 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.656277 +0000 +0000  0  NULL  NULL
        	            	858804442612006913  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:45:16.715133 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:20.843657 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.715133 +0000 +0000  0  NULL  NULL
        	            	858804442760413185  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:45:16.76042 +0000 +0000   NULL                                    NULL                                    2023-04-22 09:45:20.927306 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.76042 +0000 +0000   0  NULL  NULL
        	            	858804442895286273  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 09:45:16.801575 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:16.801575 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.801575 +0000 +0000  0  NULL  NULL
        	            	858804443047165953  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 09:45:16.847931 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:16.847931 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.847931 +0000 +0000  0  NULL  NULL
        	            	858804443192262657  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 09:45:16.892201 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:16.892201 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.892201 +0000 +0000  0  NULL  NULL
        	            	858804443351482369  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 09:45:16.940799 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:16.940799 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.940799 +0000 +0000  0  NULL  NULL
        	            	858804443542487041  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 09:45:16.999085 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:16.999085 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:16.999085 +0000 +0000  0  NULL  NULL
        	            	858804443687157761  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 09:45:17.043245 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:17.043245 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:17.043245 +0000 +0000  0  NULL  NULL
        	            	858804443833106433  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 09:45:17.087777 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:17.087777 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:17.087777 +0000 +0000  0  NULL  NULL
        	            	858804444009562113  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 09:45:17.141639 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:17.141639 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:17.141639 +0000 +0000  0  NULL  NULL
        	            	858804444156919809  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 09:45:17.186612 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:17.186612 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:17.186612 +0000 +0000  0  NULL  NULL
        	            	858804444297003009  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:45:17.229354 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:17.229354 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:17.229354 +0000 +0000  0  NULL  NULL
        	            	858804444440461313  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:45:17.273137 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:17.273137 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:17.273137 +0000 +0000  0  NULL  NULL
        	            	858804444590178305  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 09:45:17.318871 +0000 +0000  NULL                                    NULL                                    2023-04-22 09:45:17.318871 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 09:45:17.318871 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (10.65s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ e82cf4ec488258875dcf5e42e7d6745b03210edd:

        	            	859096621655490561  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-23 10:31:22.682979 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:22.682979 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:22.682979 +0000 +0000  0  NULL  NULL
        	            	859096621949648897  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 10:31:22.772783 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:22.772783 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:22.772783 +0000 +0000  0  NULL  NULL
        	            	859096622221393921  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 10:31:22.855678 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:22.855678 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:22.855678 +0000 +0000  0  NULL  NULL
        	            	859096622456569857  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 10:31:22.927444 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:22.927444 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:22.927444 +0000 +0000  0  NULL  NULL
        	            	859096622699151361  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-23 10:31:23.001473 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:23.001473 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:23.001473 +0000 +0000  0  NULL  NULL
        	            	859096622936817665  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-23 10:31:23.074019 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:23.074019 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:23.074019 +0000 +0000  0  NULL  NULL
        	            	859096623170519041  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-23 10:31:23.145324 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:23.145324 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:23.145324 +0000 +0000  0  NULL  NULL
        	            	859096623434465281  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 10:31:23.225886 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:23.225886 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:23.225886 +0000 +0000  0  NULL  NULL
        	            	859096623672360961  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 10:31:23.298477 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:23.298477 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:23.298477 +0000 +0000  0  NULL  NULL
        	            	859096623913598977  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 10:31:23.3721 +0000 +0000    NULL                                    NULL                                    2023-04-23 10:31:23.3721 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:23.3721 +0000 +0000    0  NULL  NULL
        	            	859096624142352385  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 10:31:23.441925 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:23.441925 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:23.441925 +0000 +0000  0  NULL  NULL
        	            	859096624392011777  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 10:31:23.518103 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:23.518103 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:23.518103 +0000 +0000  0  NULL  NULL
        	            	859096624630661121  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 10:31:23.59093 +0000 +0000   NULL                                    NULL                                    2023-04-23 10:31:23.59093 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:23.59093 +0000 +0000   0  NULL  NULL
        	            	859096624943824897  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 10:31:23.68649 +0000 +0000   NULL                                    NULL                                    2023-04-23 10:31:23.68649 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:23.68649 +0000 +0000   0  NULL  NULL
        	            	859096624974594049  AUTO CREATE STATS                Table statistics refresh for system.public.users                                        CREATE STATISTICS __auto__ FROM [4] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'   root      {}  succeeded         NULL  2023-04-23 10:31:23.695876 +0000 +0000  2023-04-23 10:31:23.852089 +0000 +0000  2023-04-23 10:31:24.44474 +0000 +0000   2023-04-23 10:31:24.444785 +0000 +0000  1  NULL    1     6174660649439456155  2023-04-23 10:31:23.852107 +0000 +0000  2023-04-24 10:31:23.852107 +0000 +0000  1  NULL  NULL
        	            	859096625174216705  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 10:31:23.756811 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:23.756811 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:23.756811 +0000 +0000  0  NULL  NULL
        	            	859096625423810561  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 10:31:23.832974 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:23.832974 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:23.832974 +0000 +0000  0  NULL  NULL
        	            	859096625706336257  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 10:31:23.919204 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:23.919204 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:23.919204 +0000 +0000  0  NULL  NULL
        	            	859096625955766273  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 10:31:23.995316 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:23.995316 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:23.995316 +0000 +0000  0  NULL  NULL
        	            	859096626200936449  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 10:31:24.070137 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:24.070137 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:24.070137 +0000 +0000  0  NULL  NULL
        	            	859096626437521409  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 10:31:24.142346 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:24.142346 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:24.142346 +0000 +0000  0  NULL  NULL
        	            	859096626678923265  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 10:31:24.216004 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:24.216004 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:24.216004 +0000 +0000  0  NULL  NULL
        	            	859096626920357889  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 10:31:24.289696 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:24.289696 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:24.289696 +0000 +0000  0  NULL  NULL
        	            	859096627156221953  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 10:31:24.361666 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:24.361666 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:24.361666 +0000 +0000  0  NULL  NULL
        	            	859096627396476929  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 10:31:24.434984 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:24.434984 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:24.434984 +0000 +0000  0  NULL  NULL
        	            	859096627627229185  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 10:31:24.505399 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:24.505399 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:24.505399 +0000 +0000  0  NULL  NULL
        	            	859096627853033473  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 10:31:24.574322 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:24.574322 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:24.574322 +0000 +0000  0  NULL  NULL
        	            	859096628085293057  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 10:31:24.645197 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:24.645197 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:24.645197 +0000 +0000  0  NULL  NULL
        	            	859096628319584257  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 10:31:24.716689 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:24.716689 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:24.716689 +0000 +0000  0  NULL  NULL
        	            	859096628608860161  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 10:31:24.80498 +0000 +0000   NULL                                    NULL                                    2023-04-23 10:31:24.80498 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:24.80498 +0000 +0000   0  NULL  NULL
        	            	859096628669906945  AUTO CREATE STATS                Table statistics refresh for system.public.tenants                                      CREATE STATISTICS __auto__ FROM [8] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'   root      {}  succeeded         NULL  2023-04-23 10:31:24.823596 +0000 +0000  2023-04-23 10:31:24.976843 +0000 +0000  2023-04-23 10:31:25.685347 +0000 +0000  2023-04-23 10:31:25.685398 +0000 +0000  1  NULL    1     2873309322154455789  2023-04-23 10:31:24.976865 +0000 +0000  2023-04-24 10:31:24.976865 +0000 +0000  1  NULL  NULL
        	            	859096628849541121  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 10:31:24.878429 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:24.878429 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:24.878429 +0000 +0000  0  NULL  NULL
        	            	859096629106573313  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 10:31:24.95688 +0000 +0000   NULL                                    NULL                                    2023-04-23 10:31:24.95688 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:24.95688 +0000 +0000   0  NULL  NULL
        	            	859096629349351425  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-23 10:31:25.030959 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:25.030959 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:25.030959 +0000 +0000  0  NULL  NULL
        	            	859096629585543169  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-23 10:31:25.10305 +0000 +0000   NULL                                    NULL                                    2023-04-23 10:31:25.10305 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:25.10305 +0000 +0000   0  NULL  NULL
        	            	859096629845786625  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-23 10:31:25.182473 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:25.182473 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:25.182473 +0000 +0000  0  NULL  NULL
        	            	859096630084927489  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 10:31:25.255436 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:25.255436 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:25.255436 +0000 +0000  0  NULL  NULL
        	            	859096630335307777  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 10:31:25.331841 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:25.331841 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:25.331841 +0000 +0000  0  NULL  NULL
        	            	859096630597058561  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 10:31:25.411743 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:25.411743 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:25.411743 +0000 +0000  0  NULL  NULL
        	            	859096630841507841  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 10:31:25.486327 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:25.486327 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:25.486327 +0000 +0000  0  NULL  NULL
        	            	859096631129079809  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 10:31:25.574083 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:25.574083 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:25.574083 +0000 +0000  0  NULL  NULL
        	            	859096631367335937  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 10:31:25.646795 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:25.646795 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:25.646795 +0000 +0000  0  NULL  NULL
        	            	859096631606870017  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 10:31:25.719905 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:25.719905 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:25.719905 +0000 +0000  0  NULL  NULL
        	            	859096631881072641  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 10:31:25.803578 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:25.803578 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:25.803578 +0000 +0000  0  NULL  NULL
        	            	859096632110940161  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 10:31:25.873725 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:25.873725 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:25.873725 +0000 +0000  0  NULL  NULL
        	            	859096632344543233  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 10:31:25.945024 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:25.945024 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:25.945024 +0000 +0000  0  NULL  NULL
        	            	859096632582569985  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 10:31:26.017655 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:26.017655 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:26.017655 +0000 +0000  0  NULL  NULL
        	            	859096632686968833  AUTO CREATE STATS                Table statistics refresh for system.public.ui                                           CREATE STATISTICS __auto__ FROM [14] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-23 10:31:26.0495 +0000 +0000    2023-04-23 10:31:26.196569 +0000 +0000  2023-04-23 10:31:26.776802 +0000 +0000  2023-04-23 10:31:26.776853 +0000 +0000  1  NULL    1     5501456449318384557  2023-04-23 10:31:26.196583 +0000 +0000  2023-04-24 10:31:26.196583 +0000 +0000  1  NULL  NULL
        	            	859096632823283713  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 10:31:26.091132 +0000 +0000  NULL                                    NULL                                    2023-04-23 10:31:26.091132 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 10:31:26.091132 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/cancel_ALL_RESTORE_JOBS (9.74s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1 @ e82cf4ec488258875dcf5e42e7d6745b03210edd:

        	            	859102282841260033  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-23 11:00:10.339768 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:10.339768 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:10.339768 +0000 +0000  0  NULL  NULL
        	            	859102282975182849  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-23 11:00:10.380648 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:10.380648 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:10.380648 +0000 +0000  0  NULL  NULL
        	            	859102283108909057  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-23 11:00:10.421478 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:10.421478 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:10.421478 +0000 +0000  0  NULL  NULL
        	            	859102283237949441  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 11:00:10.460863 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:10.460863 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:10.460863 +0000 +0000  0  NULL  NULL
        	            	859102283365711873  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 11:00:10.499832 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:10.499832 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:10.499832 +0000 +0000  0  NULL  NULL
        	            	859102283493408769  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 11:00:10.538798 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:10.538798 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:10.538798 +0000 +0000  0  NULL  NULL
        	            	859102283623661569  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-23 11:00:10.578549 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:10.578549 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:10.578549 +0000 +0000  0  NULL  NULL
        	            	859102283754733569  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-23 11:00:10.618552 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:10.618552 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:10.618552 +0000 +0000  0  NULL  NULL
        	            	859102283884658689  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-23 11:00:10.658194 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:10.658194 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:10.658194 +0000 +0000  0  NULL  NULL
        	            	859102284017336321  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 11:00:10.698685 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:10.698685 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:10.698685 +0000 +0000  0  NULL  NULL
        	            	859102284105351169  AUTO CREATE STATS                Table statistics refresh for system.public.role_members                                 CREATE STATISTICS __auto__ FROM [23] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-23 11:00:10.72553 +0000 +0000   2023-04-23 11:00:10.840907 +0000 +0000  2023-04-23 11:00:12.485302 +0000 +0000  2023-04-23 11:00:12.485459 +0000 +0000  1  NULL    1     291789537054491893   2023-04-23 11:00:10.840914 +0000 +0000  2023-04-24 11:00:10.840914 +0000 +0000  1  NULL  NULL
        	            	859102284151193601  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 11:00:10.73953 +0000 +0000   NULL                                    NULL                                    2023-04-23 11:00:10.73953 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:10.73953 +0000 +0000   0  NULL  NULL
        	            	859102284330926081  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 11:00:10.794386 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:10.794386 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:10.794386 +0000 +0000  0  NULL  NULL
        	            	859102284463898625  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 11:00:10.834963 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:10.834963 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:10.834963 +0000 +0000  0  NULL  NULL
        	            	859102284604571649  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 11:00:10.877895 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:10.877895 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:10.877895 +0000 +0000  0  NULL  NULL
        	            	859102284766052353  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 11:00:10.927185 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:10.927185 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:10.927185 +0000 +0000  0  NULL  NULL
        	            	859102284892241921  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 11:00:10.965691 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:10.965691 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:10.965691 +0000 +0000  0  NULL  NULL
        	            	859102285057851393  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 11:00:11.016224 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:11.016224 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.016224 +0000 +0000  0  NULL  NULL
        	            	859102285240991745  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 11:00:11.072119 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:11.072119 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.072119 +0000 +0000  0  NULL  NULL
        	            	859102285413023745  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 11:00:11.124617 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:11.124617 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.124617 +0000 +0000  0  NULL  NULL
        	            	859102285550616577  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 11:00:11.166603 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:11.166603 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.166603 +0000 +0000  0  NULL  NULL
        	            	859102285690634241  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 11:00:11.209331 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:11.209331 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.209331 +0000 +0000  0  NULL  NULL
        	            	859102285821640705  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 11:00:11.249316 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:14.853843 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.249316 +0000 +0000  0  NULL  NULL
        	            	859102285950812161  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 11:00:11.288738 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:14.923473 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.288738 +0000 +0000  0  NULL  NULL
        	            	859102286114619393  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 11:00:11.33873 +0000 +0000   NULL                                    NULL                                    2023-04-23 11:00:14.996291 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.33873 +0000 +0000   0  NULL  NULL
        	            	859102286274199553  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 11:00:11.38743 +0000 +0000   NULL                                    NULL                                    2023-04-23 11:00:15.766059 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.38743 +0000 +0000   0  NULL  NULL
        	            	859102286404714497  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 11:00:11.427259 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:15.839906 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.427259 +0000 +0000  0  NULL  NULL
        	            	859102286531854337  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 11:00:11.46605 +0000 +0000   NULL                                    NULL                                    2023-04-23 11:00:15.917235 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.46605 +0000 +0000   0  NULL  NULL
        	            	859102286657290241  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-23 11:00:11.504331 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:11.504331 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.504331 +0000 +0000  0  NULL  NULL
        	            	859102286793670657  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-23 11:00:11.545952 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:11.545952 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.545952 +0000 +0000  0  NULL  NULL
        	            	859102286919958529  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-23 11:00:11.584521 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:11.584521 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.584521 +0000 +0000  0  NULL  NULL
        	            	859102287049654273  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 11:00:11.624075 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:11.624075 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.624075 +0000 +0000  0  NULL  NULL
        	            	859102287212773377  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 11:00:11.673858 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:11.673858 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.673858 +0000 +0000  0  NULL  NULL
        	            	859102287340437505  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 11:00:11.712826 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:11.712826 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.712826 +0000 +0000  0  NULL  NULL
        	            	859102287466266625  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 11:00:11.751215 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:15.547101 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.751215 +0000 +0000  0  NULL  NULL
        	            	859102287599894529  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 11:00:11.791996 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:15.61703 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.791996 +0000 +0000  0  NULL  NULL
        	            	859102287730049025  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 11:00:11.831716 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:15.685847 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.831716 +0000 +0000  0  NULL  NULL
        	            	859102287862300673  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 11:00:11.872071 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:11.872071 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.872071 +0000 +0000  0  NULL  NULL
        	            	859102287993274369  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 11:00:11.912042 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:11.912042 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.912042 +0000 +0000  0  NULL  NULL
        	            	859102288123297793  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 11:00:11.951726 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:11.951726 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.951726 +0000 +0000  0  NULL  NULL
        	            	859102288250732545  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 11:00:11.990617 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:11.990617 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:11.990617 +0000 +0000  0  NULL  NULL
        	            	859102288379084801  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 11:00:12.029784 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:12.029784 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:12.029784 +0000 +0000  0  NULL  NULL
        	            	859102288510255105  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 11:00:12.069823 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:12.069823 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:12.069823 +0000 +0000  0  NULL  NULL
        	            	859102288645816321  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 11:00:12.111187 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:12.111187 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:12.111187 +0000 +0000  0  NULL  NULL
        	            	859102288781639681  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 11:00:12.152638 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:12.152638 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:12.152638 +0000 +0000  0  NULL  NULL
        	            	859102288934502401  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 11:00:12.199285 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:12.199285 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:12.199285 +0000 +0000  0  NULL  NULL
        	            	859102289064296449  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 11:00:12.238892 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:12.238892 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:12.238892 +0000 +0000  0  NULL  NULL
        	            	859102289198776321  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 11:00:12.279941 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:12.279941 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:12.279941 +0000 +0000  0  NULL  NULL
        	            	859102289343184897  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 11:00:12.324003 +0000 +0000  NULL                                    NULL                                    2023-04-23 11:00:12.324003 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 11:00:12.324003 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (9.95s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@yuzefovich
Copy link
Member

fixed by #101857

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-jobs
Projects
None yet
Development

No branches or pull requests

2 participants