Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jobs: TestJobControlByType failed #101294

Closed
cockroach-teamcity opened this issue Apr 12, 2023 · 22 comments
Closed

jobs: TestJobControlByType failed #101294

cockroach-teamcity opened this issue Apr 12, 2023 · 22 comments
Labels
C-test-failure Broken test (automatically or manually discovered). GA-blocker O-robot Originated from a bot. T-jobs
Milestone

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Apr 12, 2023

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ f1921dbd499fd258a606c4e7180aff7b82b6f900:

        	            	855983394646392833  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-12 10:36:41.198296 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:41.198296 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:41.198296 +0000 +0000  0  NULL  NULL
        	            	855983394858106881  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:36:41.262901 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:41.262901 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:41.262901 +0000 +0000  0  NULL  NULL
        	            	855983395072376833  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:36:41.328266 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:41.328266 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:41.328266 +0000 +0000  0  NULL  NULL
        	            	855983395074015233  AUTO CREATE STATS                Table statistics refresh for system.public.span_configurations                          CREATE STATISTICS __auto__ FROM [47] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-12 10:36:41.328755 +0000 +0000  2023-04-12 10:36:41.461137 +0000 +0000  2023-04-12 10:36:41.958814 +0000 +0000  2023-04-12 10:36:41.958863 +0000 +0000  1  NULL    1     7204407852816238060  2023-04-12 10:36:41.461146 +0000 +0000  2023-04-13 10:36:41.461146 +0000 +0000  1  NULL  NULL
        	            	855983395297394689  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:36:41.39695 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:36:41.39695 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:41.39695 +0000 +0000   0  NULL  NULL
        	            	855983395499868161  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-12 10:36:41.45873 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:36:41.45873 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:41.45873 +0000 +0000   0  NULL  NULL
        	            	855983395704471553  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-12 10:36:41.52117 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:36:41.52117 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:41.52117 +0000 +0000   0  NULL  NULL
        	            	855983395929522177  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-12 10:36:41.589844 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:41.589844 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:41.589844 +0000 +0000  0  NULL  NULL
        	            	855983396143169537  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:36:41.65505 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:36:41.65505 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:41.65505 +0000 +0000   0  NULL  NULL
        	            	855983396353146881  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:36:41.719129 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:41.719129 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:41.719129 +0000 +0000  0  NULL  NULL
        	            	855983396559388673  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:36:41.782065 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:41.782065 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:41.782065 +0000 +0000  0  NULL  NULL
        	            	855983396767825921  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:36:41.845673 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:41.845673 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:41.845673 +0000 +0000  0  NULL  NULL
        	            	855983396970725377  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:36:41.907605 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:41.907605 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:41.907605 +0000 +0000  0  NULL  NULL
        	            	855983397185224705  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:36:41.973057 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:41.973057 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:41.973057 +0000 +0000  0  NULL  NULL
        	            	855983397392941057  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:36:42.036444 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:42.036444 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:42.036444 +0000 +0000  0  NULL  NULL
        	            	855983397616156673  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:36:42.104562 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:42.104562 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:42.104562 +0000 +0000  0  NULL  NULL
        	            	855983397828493313  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:36:42.169366 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:42.169366 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:42.169366 +0000 +0000  0  NULL  NULL
        	            	855983398039191553  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:36:42.233664 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:42.233664 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:42.233664 +0000 +0000  0  NULL  NULL
        	            	855983398157680641  AUTO CREATE STATS                Table statistics refresh for system.public.replication_critical_localities              CREATE STATISTICS __auto__ FROM [26] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-12 10:36:42.269808 +0000 +0000  2023-04-12 10:36:42.416636 +0000 +0000  2023-04-12 10:36:43.14075 +0000 +0000   2023-04-12 10:36:43.140801 +0000 +0000  1  NULL    1     4407596333825934708  2023-04-12 10:36:42.416652 +0000 +0000  2023-04-13 10:36:42.416652 +0000 +0000  1  NULL  NULL
        	            	855983398242746369  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:36:42.295779 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:42.295779 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:42.295779 +0000 +0000  0  NULL  NULL
        	            	855983398456918017  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:36:42.361145 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:42.361145 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:42.361145 +0000 +0000  0  NULL  NULL
        	            	855983398691930113  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:36:42.432862 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:45.2729 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:42.432862 +0000 +0000  0  NULL  NULL
        	            	855983398896893953  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:36:42.49541 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:36:45.347926 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:42.49541 +0000 +0000   0  NULL  NULL
        	            	855983399101825025  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:36:42.557965 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:45.425148 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:42.557965 +0000 +0000  0  NULL  NULL
        	            	855983399332839425  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:36:42.628463 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:45.792783 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:42.628463 +0000 +0000  0  NULL  NULL
        	            	855983399543078913  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:36:42.692621 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:45.871252 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:42.692621 +0000 +0000  0  NULL  NULL
        	            	855983399776485377  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:36:42.763852 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:45.947921 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:42.763852 +0000 +0000  0  NULL  NULL
        	            	855983400016510977  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-12 10:36:42.837096 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:42.837096 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:42.837096 +0000 +0000  0  NULL  NULL
        	            	855983400246312961  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-12 10:36:42.907221 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:42.907221 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:42.907221 +0000 +0000  0  NULL  NULL
        	            	855983400455438337  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-12 10:36:42.971057 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:42.971057 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:42.971057 +0000 +0000  0  NULL  NULL
        	            	855983400657846273  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:36:43.032818 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:43.032818 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:43.032818 +0000 +0000  0  NULL  NULL
        	            	855983400873984001  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:36:43.098792 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:43.098792 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:43.098792 +0000 +0000  0  NULL  NULL
        	            	855983401099067393  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:36:43.167461 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:43.167461 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:43.167461 +0000 +0000  0  NULL  NULL
        	            	855983401299640321  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:36:43.228676 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:45.553173 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:43.228676 +0000 +0000  0  NULL  NULL
        	            	855983401502605313  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:36:43.290628 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:45.632071 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:43.290628 +0000 +0000  0  NULL  NULL
        	            	855983401696133121  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:36:43.349684 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:45.7097 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:43.349684 +0000 +0000  0  NULL  NULL
        	            	855983401920659457  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:36:43.418261 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:43.418261 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:43.418261 +0000 +0000  0  NULL  NULL
        	            	855983402105831425  AUTO CREATE STATS                Table statistics refresh for system.public.protected_ts_meta                            CREATE STATISTICS __auto__ FROM [31] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-12 10:36:43.474697 +0000 +0000  2023-04-12 10:36:43.608863 +0000 +0000  2023-04-12 10:36:44.237048 +0000 +0000  2023-04-12 10:36:44.237098 +0000 +0000  1  NULL    1     3344588766151421487  2023-04-12 10:36:43.608878 +0000 +0000  2023-04-13 10:36:43.608878 +0000 +0000  1  NULL  NULL
        	            	855983402120839169  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:36:43.479292 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:43.479292 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:43.479292 +0000 +0000  0  NULL  NULL
        	            	855983402325245953  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:36:43.541673 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:43.541673 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:43.541673 +0000 +0000  0  NULL  NULL
        	            	855983402530766849  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:36:43.604387 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:43.604387 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:43.604387 +0000 +0000  0  NULL  NULL
        	            	855983402736091137  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:36:43.667044 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:43.667044 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:43.667044 +0000 +0000  0  NULL  NULL
        	            	855983402933059585  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:36:43.727166 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:43.727166 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:43.727166 +0000 +0000  0  NULL  NULL
        	            	855983403137368065  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:36:43.789508 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:43.789508 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:43.789508 +0000 +0000  0  NULL  NULL
        	            	855983403336433665  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:36:43.85026 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:36:43.85026 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:43.85026 +0000 +0000   0  NULL  NULL
        	            	855983403541790721  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:36:43.912924 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:43.912924 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:43.912924 +0000 +0000  0  NULL  NULL
        	            	855983403755765761  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:36:43.978228 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:43.978228 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:43.978228 +0000 +0000  0  NULL  NULL
        	            	855983403988549633  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:36:44.04927 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:36:44.04927 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:44.04927 +0000 +0000   0  NULL  NULL
        	            	855983404229754881  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:36:44.122899 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:36:44.122899 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:36:44.122899 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (9.16s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

/cc @cockroachdb/jobs

This test on roachdash | Improve this report!

Jira issue: CRDB-27035

@cockroach-teamcity cockroach-teamcity added branch-release-23.1.0 C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Apr 12, 2023
@cockroach-teamcity cockroach-teamcity added this to the 23.1 milestone Apr 12, 2023
@blathers-crl blathers-crl bot added the T-jobs label Apr 12, 2023
@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ f1921dbd499fd258a606c4e7180aff7b82b6f900:

        	            	855986623476498433  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-12 10:53:06.559024 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:06.559024 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:06.559024 +0000 +0000  0  NULL  NULL
        	            	855986623545147393  AUTO CREATE STATS                Table statistics refresh for system.public.migrations                                   CREATE STATISTICS __auto__ FROM [40] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-12 10:53:06.579953 +0000 +0000  2023-04-12 10:53:06.672283 +0000 +0000  2023-04-12 10:53:08.467919 +0000 +0000  2023-04-12 10:53:08.468023 +0000 +0000  1  NULL    1     2493021830317806937  2023-04-12 10:53:06.672289 +0000 +0000  2023-04-13 10:53:06.672289 +0000 +0000  1  NULL  NULL
        	            	855986623595708417  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-12 10:53:06.595404 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:06.595404 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:06.595404 +0000 +0000  0  NULL  NULL
        	            	855986623714656257  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-12 10:53:06.631703 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:06.631703 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:06.631703 +0000 +0000  0  NULL  NULL
        	            	855986623839469569  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:53:06.669799 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:06.669799 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:06.669799 +0000 +0000  0  NULL  NULL
        	            	855986623970213889  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:53:06.709692 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:06.709692 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:06.709692 +0000 +0000  0  NULL  NULL
        	            	855986624113311745  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:53:06.753363 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:06.753363 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:06.753363 +0000 +0000  0  NULL  NULL
        	            	855986624229834753  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-12 10:53:06.788934 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:06.788934 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:06.788934 +0000 +0000  0  NULL  NULL
        	            	855986624346226689  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-12 10:53:06.824437 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:06.824437 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:06.824437 +0000 +0000  0  NULL  NULL
        	            	855986624463568897  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-12 10:53:06.860243 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:06.860243 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:06.860243 +0000 +0000  0  NULL  NULL
        	            	855986624580452353  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:53:06.895925 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:06.895925 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:06.895925 +0000 +0000  0  NULL  NULL
        	            	855986624696385537  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:53:06.931314 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:06.931314 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:06.931314 +0000 +0000  0  NULL  NULL
        	            	855986624859897857  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:53:06.981211 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:06.981211 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:06.981211 +0000 +0000  0  NULL  NULL
        	            	855986625004699649  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:53:07.025391 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.025391 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.025391 +0000 +0000  0  NULL  NULL
        	            	855986625157595137  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:53:07.072051 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.072051 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.072051 +0000 +0000  0  NULL  NULL
        	            	855986625273561089  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:53:07.107442 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.107442 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.107442 +0000 +0000  0  NULL  NULL
        	            	855986625392345089  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:53:07.143687 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.143687 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.143687 +0000 +0000  0  NULL  NULL
        	            	855986625512177665  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:53:07.180266 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.180266 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.180266 +0000 +0000  0  NULL  NULL
        	            	855986625636859905  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:53:07.218315 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.218315 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.218315 +0000 +0000  0  NULL  NULL
        	            	855986625763377153  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:53:07.256918 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.256918 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.256918 +0000 +0000  0  NULL  NULL
        	            	855986625883635713  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:53:07.293616 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.293616 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.293616 +0000 +0000  0  NULL  NULL
        	            	855986625998815233  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:53:07.328766 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.328766 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.328766 +0000 +0000  0  NULL  NULL
        	            	855986626114453505  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-12 10:53:07.364056 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.364056 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.364056 +0000 +0000  0  NULL  NULL
        	            	855986626237857793  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-12 10:53:07.401716 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.401716 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.401716 +0000 +0000  0  NULL  NULL
        	            	855986626354642945  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-12 10:53:07.437374 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.437374 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.437374 +0000 +0000  0  NULL  NULL
        	            	855986626480242689  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-12 10:53:07.475687 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.475687 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.475687 +0000 +0000  0  NULL  NULL
        	            	855986626626093057  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-12 10:53:07.520208 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.520208 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.520208 +0000 +0000  0  NULL  NULL
        	            	855986626744713217  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-12 10:53:07.556399 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.556399 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.556399 +0000 +0000  0  NULL  NULL
        	            	855986626860384257  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-12 10:53:07.591717 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.591717 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.591717 +0000 +0000  0  NULL  NULL
        	            	855986626978807809  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-12 10:53:07.627841 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.627841 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.627841 +0000 +0000  0  NULL  NULL
        	            	855986627095068673  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-12 10:53:07.663326 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.663326 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.663326 +0000 +0000  0  NULL  NULL
        	            	855986627235840001  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:53:07.706291 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.706291 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.706291 +0000 +0000  0  NULL  NULL
        	            	855986627349807105  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:53:07.741061 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.741061 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.741061 +0000 +0000  0  NULL  NULL
        	            	855986627466788865  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-12 10:53:07.776765 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.776765 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.776765 +0000 +0000  0  NULL  NULL
        	            	855986627581607937  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-12 10:53:07.811802 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.811802 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.811802 +0000 +0000  0  NULL  NULL
        	            	855986627701309441  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-12 10:53:07.848338 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.848338 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.848338 +0000 +0000  0  NULL  NULL
        	            	855986627820322817  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-12 10:53:07.884667 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.884667 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.884667 +0000 +0000  0  NULL  NULL
        	            	855986627941302273  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:53:07.921572 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.921572 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.921572 +0000 +0000  0  NULL  NULL
        	            	855986628058972161  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:53:07.957487 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.957487 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.957487 +0000 +0000  0  NULL  NULL
        	            	855986628175822849  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-12 10:53:07.993143 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:07.993143 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:07.993143 +0000 +0000  0  NULL  NULL
        	            	855986628289855489  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:53:08.027935 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:08.027935 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:08.027935 +0000 +0000  0  NULL  NULL
        	            	855986628405592065  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:53:08.063265 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:08.063265 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:08.063265 +0000 +0000  0  NULL  NULL
        	            	855986628522278913  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-12 10:53:08.098872 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:08.098872 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:08.098872 +0000 +0000  0  NULL  NULL
        	            	855986628643848193  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:53:08.135975 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:08.135975 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:08.135975 +0000 +0000  0  NULL  NULL
        	            	855986628759814145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:53:08.171367 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:08.171367 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:08.171367 +0000 +0000  0  NULL  NULL
        	            	855986628904878081  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-12 10:53:08.215631 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:08.215631 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:08.215631 +0000 +0000  0  NULL  NULL
        	            	855986629037228033  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:53:08.25603 +0000 +0000   NULL                                    NULL                                    2023-04-12 10:53:08.25603 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:08.25603 +0000 +0000   0  NULL  NULL
        	            	855986629201625089  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:53:08.306199 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:08.306199 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:08.306199 +0000 +0000  0  NULL  NULL
        	            	855986629327585281  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-12 10:53:08.344662 +0000 +0000  NULL                                    NULL                                    2023-04-12 10:53:08.344662 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-12 10:53:08.344662 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (6.75s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ 4db6f5ede046aa6cbc9e487dedd9e2941ef71d82:

        	            	856268810801184769  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:48:23.296285 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:23.296285 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:23.296285 +0000 +0000  0  NULL  NULL
        	            	856268811053137921  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:48:23.373177 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:23.373177 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:23.373177 +0000 +0000  0  NULL  NULL
        	            	856268811283595265  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:48:23.443514 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:23.443514 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:23.443514 +0000 +0000  0  NULL  NULL
        	            	856268811514314753  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:48:23.513921 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:23.513921 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:23.513921 +0000 +0000  0  NULL  NULL
        	            	856268811744739329  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:48:23.584234 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:23.584234 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:23.584234 +0000 +0000  0  NULL  NULL
        	            	856268811962089473  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-13 10:48:23.650574 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:23.650574 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:23.650574 +0000 +0000  0  NULL  NULL
        	            	856268812176850945  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-13 10:48:23.716111 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:23.716111 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:23.716111 +0000 +0000  0  NULL  NULL
        	            	856268812395347969  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-13 10:48:23.782805 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:23.782805 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:23.782805 +0000 +0000  0  NULL  NULL
        	            	856268812610273281  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:48:23.84839 +0000 +0000   NULL                                    NULL                                    2023-04-13 10:48:23.84839 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:23.84839 +0000 +0000   0  NULL  NULL
        	            	856268812834504705  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:48:23.916812 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:23.916812 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:23.916812 +0000 +0000  0  NULL  NULL
        	            	856268813091405825  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:48:23.995207 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:23.995207 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:23.995207 +0000 +0000  0  NULL  NULL
        	            	856268813310459905  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:48:24.062062 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:24.062062 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:24.062062 +0000 +0000  0  NULL  NULL
        	            	856268813547175937  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:48:24.134302 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:24.134302 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:24.134302 +0000 +0000  0  NULL  NULL
        	            	856268813755842561  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:48:24.197982 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:24.197982 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:24.197982 +0000 +0000  0  NULL  NULL
        	            	856268813975322625  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:48:24.264962 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:24.264962 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:24.264962 +0000 +0000  0  NULL  NULL
        	            	856268814204534785  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:48:24.334915 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:24.334915 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:24.334915 +0000 +0000  0  NULL  NULL
        	            	856268814418640897  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:48:24.400244 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:24.400244 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:24.400244 +0000 +0000  0  NULL  NULL
        	            	856268814479785985  AUTO CREATE STATS                Table statistics refresh for system.public.zones                                        CREATE STATISTICS __auto__ FROM [5] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'   root      {}  succeeded         NULL  2023-04-13 10:48:24.418893 +0000 +0000  2023-04-13 10:48:24.559444 +0000 +0000  2023-04-13 10:48:25.329774 +0000 +0000  2023-04-13 10:48:25.329825 +0000 +0000  1  NULL    1     2966215580272580709  2023-04-13 10:48:24.559464 +0000 +0000  2023-04-14 10:48:24.559464 +0000 +0000  1  NULL  NULL
        	            	856268814633664513  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:48:24.465865 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:24.465865 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:24.465865 +0000 +0000  0  NULL  NULL
        	            	856268814846427137  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:48:24.530812 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:24.530812 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:24.530812 +0000 +0000  0  NULL  NULL
        	            	856268815061385217  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:48:24.596395 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:24.596395 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:24.596395 +0000 +0000  0  NULL  NULL
        	            	856268815293251585  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:48:24.667161 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:27.572626 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:24.667161 +0000 +0000  0  NULL  NULL
        	            	856268815506735105  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:48:24.732317 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:27.652932 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:24.732317 +0000 +0000  0  NULL  NULL
        	            	856268815721398273  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:48:24.797822 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:27.733348 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:24.797822 +0000 +0000  0  NULL  NULL
        	            	856268815927181313  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:48:24.860632 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:28.121971 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:24.860632 +0000 +0000  0  NULL  NULL
        	            	856268816211804161  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:48:24.947508 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:28.213107 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:24.947508 +0000 +0000  0  NULL  NULL
        	            	856268816424173569  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:48:25.012288 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:28.296707 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:25.012288 +0000 +0000  0  NULL  NULL
        	            	856268816751427585  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:48:25.112153 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:25.112153 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:25.112153 +0000 +0000  0  NULL  NULL
        	            	856268816964321281  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:48:25.177136 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:25.177136 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:25.177136 +0000 +0000  0  NULL  NULL
        	            	856268817171546113  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:48:25.240369 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:25.240369 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:25.240369 +0000 +0000  0  NULL  NULL
        	            	856268817389518849  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:48:25.306885 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:25.306885 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:25.306885 +0000 +0000  0  NULL  NULL
        	            	856268817597038593  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:48:25.370219 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:25.370219 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:25.370219 +0000 +0000  0  NULL  NULL
        	            	856268817806426113  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:48:25.434123 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:25.434123 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:25.434123 +0000 +0000  0  NULL  NULL
        	            	856268818017681409  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:48:25.498592 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:27.865119 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:25.498592 +0000 +0000  0  NULL  NULL
        	            	856268818233884673  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:48:25.564584 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:27.949435 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:25.564584 +0000 +0000  0  NULL  NULL
        	            	856268818452611073  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:48:25.631318 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:28.032018 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:25.631318 +0000 +0000  0  NULL  NULL
        	            	856268818551832577  AUTO CREATE STATS                Table statistics refresh for system.public.rangelog                                     CREATE STATISTICS __auto__ FROM [13] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-13 10:48:25.661587 +0000 +0000  2023-04-13 10:48:25.801975 +0000 +0000  2023-04-13 10:48:26.531671 +0000 +0000  2023-04-13 10:48:26.531726 +0000 +0000  1  NULL    1     1306456394379888983  2023-04-13 10:48:25.801989 +0000 +0000  2023-04-14 10:48:25.801989 +0000 +0000  1  NULL  NULL
        	            	856268818670780417  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:48:25.697904 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:25.697904 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:25.697904 +0000 +0000  0  NULL  NULL
        	            	856268818878103553  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:48:25.76117 +0000 +0000   NULL                                    NULL                                    2023-04-13 10:48:25.76117 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:25.76117 +0000 +0000   0  NULL  NULL
        	            	856268819089752065  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:48:25.82576 +0000 +0000   NULL                                    NULL                                    2023-04-13 10:48:25.82576 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:25.82576 +0000 +0000   0  NULL  NULL
        	            	856268819297304577  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:48:25.88911 +0000 +0000   NULL                                    NULL                                    2023-04-13 10:48:25.88911 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:25.88911 +0000 +0000   0  NULL  NULL
        	            	856268819500171265  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:48:25.951013 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:25.951013 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:25.951013 +0000 +0000  0  NULL  NULL
        	            	856268819753598977  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:48:26.028355 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:26.028355 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:26.028355 +0000 +0000  0  NULL  NULL
        	            	856268819987922945  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:48:26.099872 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:26.099872 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:26.099872 +0000 +0000  0  NULL  NULL
        	            	856268820203110401  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:48:26.165532 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:26.165532 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:26.165532 +0000 +0000  0  NULL  NULL
        	            	856268820430028801  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:48:26.234811 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:26.234811 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:26.234811 +0000 +0000  0  NULL  NULL
        	            	856268820662550529  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:48:26.305741 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:26.305741 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:26.305741 +0000 +0000  0  NULL  NULL
        	            	856268820875870209  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:48:26.370848 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:26.370848 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:26.370848 +0000 +0000  0  NULL  NULL
        	            	856268821086208001  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:48:26.435025 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:48:26.435025 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:48:26.435025 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (9.59s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ 4db6f5ede046aa6cbc9e487dedd9e2941ef71d82:

        	            	856269324384763905  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:51:00.029578 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.029578 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.029578 +0000 +0000  0  NULL  NULL
        	            	856269324523995137  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:51:00.072059 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.072059 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.072059 +0000 +0000  0  NULL  NULL
        	            	856269324676169729  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:51:00.118501 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.118501 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.118501 +0000 +0000  0  NULL  NULL
        	            	856269324826836993  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:51:00.164485 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.164485 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.164485 +0000 +0000  0  NULL  NULL
        	            	856269325043761153  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:51:00.230688 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.230688 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.230688 +0000 +0000  0  NULL  NULL
        	            	856269325326155777  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:51:00.316868 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.316868 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.316868 +0000 +0000  0  NULL  NULL
        	            	856269325540229121  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-13 10:51:00.382199 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.382199 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.382199 +0000 +0000  0  NULL  NULL
        	            	856269325647806465  AUTO CREATE STATS                Table statistics refresh for system.public.users                                        CREATE STATISTICS __auto__ FROM [4] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'   root      {}  succeeded         NULL  2023-04-13 10:51:00.415011 +0000 +0000  2023-04-13 10:51:00.527287 +0000 +0000  2023-04-13 10:51:02.631388 +0000 +0000  2023-04-13 10:51:02.631537 +0000 +0000  1  NULL    1     2987183437883365727  2023-04-13 10:51:00.527293 +0000 +0000  2023-04-14 10:51:00.527293 +0000 +0000  1  NULL  NULL
        	            	856269325699153921  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-13 10:51:00.430722 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.430722 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.430722 +0000 +0000  0  NULL  NULL
        	            	856269325838974977  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-13 10:51:00.473364 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.473364 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.473364 +0000 +0000  0  NULL  NULL
        	            	856269325984497665  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:51:00.517784 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.517784 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.517784 +0000 +0000  0  NULL  NULL
        	            	856269326126579713  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:51:00.561119 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.561119 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.561119 +0000 +0000  0  NULL  NULL
        	            	856269326332493825  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:51:00.623975 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.623975 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.623975 +0000 +0000  0  NULL  NULL
        	            	856269326483292161  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:51:00.669991 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.669991 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.669991 +0000 +0000  0  NULL  NULL
        	            	856269326621900801  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:51:00.712301 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.712301 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.712301 +0000 +0000  0  NULL  NULL
        	            	856269326800650241  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:51:00.766844 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.766844 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.766844 +0000 +0000  0  NULL  NULL
        	            	856269326941847553  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:51:00.809936 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.809936 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.809936 +0000 +0000  0  NULL  NULL
        	            	856269327087271937  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:51:00.854304 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.854304 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.854304 +0000 +0000  0  NULL  NULL
        	            	856269327270903809  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:51:00.910358 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.910358 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.910358 +0000 +0000  0  NULL  NULL
        	            	856269327535669249  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:00.991149 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:00.991149 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:00.991149 +0000 +0000  0  NULL  NULL
        	            	856269327704752129  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:01.042747 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:01.042747 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.042747 +0000 +0000  0  NULL  NULL
        	            	856269327877537793  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:01.095477 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:01.095477 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.095477 +0000 +0000  0  NULL  NULL
        	            	856269328017391617  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:01.138159 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:06.523315 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.138159 +0000 +0000  0  NULL  NULL
        	            	856269328163176449  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:01.182647 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:06.595924 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.182647 +0000 +0000  0  NULL  NULL
        	            	856269328321806337  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:01.231087 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:06.672422 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.231087 +0000 +0000  0  NULL  NULL
        	            	856269328518184961  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:01.290994 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:07.531678 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.290994 +0000 +0000  0  NULL  NULL
        	            	856269328643719169  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:01.329297 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:07.601425 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.329297 +0000 +0000  0  NULL  NULL
        	            	856269328791601153  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:01.374426 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:07.670599 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.374426 +0000 +0000  0  NULL  NULL
        	            	856269328919396353  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:51:01.413433 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:01.413433 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.413433 +0000 +0000  0  NULL  NULL
        	            	856269329060265985  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:51:01.456429 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:01.456429 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.456429 +0000 +0000  0  NULL  NULL
        	            	856269329199300609  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-13 10:51:01.498867 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:01.498867 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.498867 +0000 +0000  0  NULL  NULL
        	            	856269329329389569  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:51:01.538554 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:01.538554 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.538554 +0000 +0000  0  NULL  NULL
        	            	856269329463181313  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:51:01.579392 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:01.579392 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.579392 +0000 +0000  0  NULL  NULL
        	            	856269329594318849  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-13 10:51:01.619414 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:01.619414 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.619414 +0000 +0000  0  NULL  NULL
        	            	856269329731878913  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:01.6614 +0000 +0000    NULL                                    NULL                                    2023-04-13 10:51:07.310778 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.6614 +0000 +0000    0  NULL  NULL
        	            	856269329894309889  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:01.710943 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:07.381048 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.710943 +0000 +0000  0  NULL  NULL
        	            	856269330027937793  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:01.751735 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:07.45337 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.751735 +0000 +0000  0  NULL  NULL
        	            	856269330179293185  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:51:01.79794 +0000 +0000   NULL                                    NULL                                    2023-04-13 10:51:01.79794 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.79794 +0000 +0000   0  NULL  NULL
        	            	856269330321309697  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:51:01.841261 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:01.841261 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.841261 +0000 +0000  0  NULL  NULL
        	            	856269330460475393  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-13 10:51:01.883736 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:01.883736 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.883736 +0000 +0000  0  NULL  NULL
        	            	856269330595807233  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:51:01.925029 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:01.925029 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.925029 +0000 +0000  0  NULL  NULL
        	            	856269330735661057  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:51:01.967717 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:01.967717 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:01.967717 +0000 +0000  0  NULL  NULL
        	            	856269330871418881  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-13 10:51:02.009138 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:02.009138 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:02.009138 +0000 +0000  0  NULL  NULL
        	            	856269331003473921  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:51:02.049446 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:02.049446 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:02.049446 +0000 +0000  0  NULL  NULL
        	            	856269331142803457  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:51:02.091957 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:02.091957 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:02.091957 +0000 +0000  0  NULL  NULL
        	            	856269331309363201  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-13 10:51:02.142797 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:02.142797 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:02.142797 +0000 +0000  0  NULL  NULL
        	            	856269331447775233  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:02.185041 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:02.185041 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:02.185041 +0000 +0000  0  NULL  NULL
        	            	856269331599917057  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:02.231484 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:02.231484 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:02.231484 +0000 +0000  0  NULL  NULL
        	            	856269331754713089  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-13 10:51:02.278701 +0000 +0000  NULL                                    NULL                                    2023-04-13 10:51:02.278701 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-13 10:51:02.278701 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (11.88s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ 5e9f0c062e6b166b76148f7a6e10c2f509658d37:

        	            	856536202390274049  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-14 09:28:24.733393 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:24.733393 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:24.733393 +0000 +0000  0  NULL  NULL
        	            	856536202575511553  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-14 09:28:24.789915 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:24.789915 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:24.789915 +0000 +0000  0  NULL  NULL
        	            	856536202793844737  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-14 09:28:24.856559 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:24.856559 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:24.856559 +0000 +0000  0  NULL  NULL
        	            	856536202842734593  AUTO CREATE STATS                Table statistics refresh for system.public.eventlog                                     CREATE STATISTICS __auto__ FROM [12] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-14 09:28:24.871451 +0000 +0000  2023-04-14 09:28:24.99973 +0000 +0000   2023-04-14 09:28:25.826634 +0000 +0000  2023-04-14 09:28:25.826681 +0000 +0000  1  NULL    1     7364335422904419215  2023-04-14 09:28:24.999739 +0000 +0000  2023-04-15 09:28:24.999739 +0000 +0000  1  NULL  NULL
        	            	856536203008933889  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-14 09:28:24.922185 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:24.922185 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:24.922185 +0000 +0000  0  NULL  NULL
        	            	856536203196104705  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-14 09:28:24.97934 +0000 +0000   NULL                                    NULL                                    2023-04-14 09:28:24.97934 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:24.97934 +0000 +0000   0  NULL  NULL
        	            	856536203385733121  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-14 09:28:25.037177 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:25.037177 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:25.037177 +0000 +0000  0  NULL  NULL
        	            	856536203586666497  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-14 09:28:25.098499 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:25.098499 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:25.098499 +0000 +0000  0  NULL  NULL
        	            	856536203774525441  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-14 09:28:25.155834 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:25.155834 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:25.155834 +0000 +0000  0  NULL  NULL
        	            	856536203979554817  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-14 09:28:25.218398 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:25.218398 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:25.218398 +0000 +0000  0  NULL  NULL
        	            	856536204237438977  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-14 09:28:25.297096 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:25.297096 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:25.297096 +0000 +0000  0  NULL  NULL
        	            	856536204425822209  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-14 09:28:25.354586 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:25.354586 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:25.354586 +0000 +0000  0  NULL  NULL
        	            	856536204611715073  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-14 09:28:25.411316 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:25.411316 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:25.411316 +0000 +0000  0  NULL  NULL
        	            	856536204802949121  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-14 09:28:25.469683 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:25.469683 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:25.469683 +0000 +0000  0  NULL  NULL
        	            	856536204991725569  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-14 09:28:25.527276 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:25.527276 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:25.527276 +0000 +0000  0  NULL  NULL
        	            	856536205185056769  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-14 09:28:25.586279 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:25.586279 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:25.586279 +0000 +0000  0  NULL  NULL
        	            	856536205405913089  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-14 09:28:25.653692 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:25.653692 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:25.653692 +0000 +0000  0  NULL  NULL
        	            	856536205586726913  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:28:25.708865 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:25.708865 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:25.708865 +0000 +0000  0  NULL  NULL
        	            	856536205769441281  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:28:25.764622 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:25.764622 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:25.764622 +0000 +0000  0  NULL  NULL
        	            	856536205954547713  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:28:25.821112 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:25.821112 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:25.821112 +0000 +0000  0  NULL  NULL
        	            	856536206150303745  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:28:25.880859 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:28.441675 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:25.880859 +0000 +0000  0  NULL  NULL
        	            	856536206339833857  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:28:25.938702 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:28.510954 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:25.938702 +0000 +0000  0  NULL  NULL
        	            	856536206561804289  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:28:26.006456 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:28.58379 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:26.006456 +0000 +0000  0  NULL  NULL
        	            	856536206742224897  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:28:26.061495 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:28.905484 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:26.061495 +0000 +0000  0  NULL  NULL
        	            	856536206920220673  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:28:26.115811 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:28.973914 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:26.115811 +0000 +0000  0  NULL  NULL
        	            	856536207025700865  AUTO CREATE STATS                Table statistics refresh for system.public.migrations                                   CREATE STATISTICS __auto__ FROM [40] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-14 09:28:26.147994 +0000 +0000  2023-04-14 09:28:26.275352 +0000 +0000  2023-04-14 09:28:26.928954 +0000 +0000  2023-04-14 09:28:26.928989 +0000 +0000  1  NULL    1     7033203766337159201  2023-04-14 09:28:26.275367 +0000 +0000  2023-04-15 09:28:26.275367 +0000 +0000  1  NULL  NULL
        	            	856536207099625473  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:28:26.170568 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:29.04385 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:26.170568 +0000 +0000  0  NULL  NULL
        	            	856536207286435841  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-14 09:28:26.22758 +0000 +0000   NULL                                    NULL                                    2023-04-14 09:28:26.22758 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:26.22758 +0000 +0000   0  NULL  NULL
        	            	856536207490875393  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-14 09:28:26.289969 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:26.289969 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:26.289969 +0000 +0000  0  NULL  NULL
        	            	856536207686533121  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-14 09:28:26.349681 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:26.349681 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:26.349681 +0000 +0000  0  NULL  NULL
        	            	856536207875375105  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-14 09:28:26.407305 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:26.407305 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:26.407305 +0000 +0000  0  NULL  NULL
        	            	856536208052224001  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-14 09:28:26.461274 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:26.461274 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:26.461274 +0000 +0000  0  NULL  NULL
        	            	856536208238542849  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-14 09:28:26.518138 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:26.518138 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:26.518138 +0000 +0000  0  NULL  NULL
        	            	856536208425222145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:28:26.575103 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:28.688797 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:26.575103 +0000 +0000  0  NULL  NULL
        	            	856536208617111553  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:28:26.633662 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:28.762803 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:26.633662 +0000 +0000  0  NULL  NULL
        	            	856536208809459713  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:28:26.692369 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:28.830052 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:26.692369 +0000 +0000  0  NULL  NULL
        	            	856536208990109697  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-14 09:28:26.747495 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:26.747495 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:26.747495 +0000 +0000  0  NULL  NULL
        	            	856536209207853057  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-14 09:28:26.813944 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:26.813944 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:26.813944 +0000 +0000  0  NULL  NULL
        	            	856536209402560513  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-14 09:28:26.873376 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:26.873376 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:26.873376 +0000 +0000  0  NULL  NULL
        	            	856536209603067905  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-14 09:28:26.934551 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:26.934551 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:26.934551 +0000 +0000  0  NULL  NULL
        	            	856536209787551745  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-14 09:28:26.990855 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:26.990855 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:26.990855 +0000 +0000  0  NULL  NULL
        	            	856536210018762753  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-14 09:28:27.061434 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:27.061434 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:27.061434 +0000 +0000  0  NULL  NULL
        	            	856536210231820289  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-14 09:28:27.12644 +0000 +0000   NULL                                    NULL                                    2023-04-14 09:28:27.12644 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:27.12644 +0000 +0000   0  NULL  NULL
        	            	856536210426920961  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-14 09:28:27.185977 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:27.185977 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:27.185977 +0000 +0000  0  NULL  NULL
        	            	856536210622349313  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-14 09:28:27.245625 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:27.245625 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:27.245625 +0000 +0000  0  NULL  NULL
        	            	856536210808504321  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:28:27.302428 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:27.302428 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:27.302428 +0000 +0000  0  NULL  NULL
        	            	856536210996133889  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:28:27.359694 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:27.359694 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:27.359694 +0000 +0000  0  NULL  NULL
        	            	856536211188285441  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:28:27.418337 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:28:27.418337 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:28:27.418337 +0000 +0000  0  NULL  NULL
        	            	856536211297140737  AUTO CREATE STATS                Table statistics refresh for system.public.settings                                     CREATE STATISTICS __auto__ FROM [6] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'   root      {}  running           NULL  2023-04-14 09:28:27.451534 +0000 +0000  2023-04-14 09:28:27.568863 +0000 +0000  NULL                                    2023-04-14 09:28:27.526757 +0000 +0000  0  NULL    1     768080486957009878   2023-04-14 09:28:27.568878 +0000 +0000  2023-04-15 09:28:27.568878 +0000 +0000  1  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (7.72s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ 5e9f0c062e6b166b76148f7a6e10c2f509658d37:

        	            	856539594501324801  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-14 09:45:39.923537 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:39.923537 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:39.923537 +0000 +0000  0  NULL  NULL
        	            	856539594636460033  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-14 09:45:39.964785 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:39.964785 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:39.964785 +0000 +0000  0  NULL  NULL
        	            	856539594760454145  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-14 09:45:40.002617 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:40.002617 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.002617 +0000 +0000  0  NULL  NULL
        	            	856539594881728513  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-14 09:45:40.039623 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:40.039623 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.039623 +0000 +0000  0  NULL  NULL
        	            	856539595001397249  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-14 09:45:40.076145 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:40.076145 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.076145 +0000 +0000  0  NULL  NULL
        	            	856539595124310017  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-14 09:45:40.11366 +0000 +0000   NULL                                    NULL                                    2023-04-14 09:45:40.11366 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.11366 +0000 +0000   0  NULL  NULL
        	            	856539595240570881  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-14 09:45:40.149134 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:40.149134 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.149134 +0000 +0000  0  NULL  NULL
        	            	856539595368071169  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-14 09:45:40.18805 +0000 +0000   NULL                                    NULL                                    2023-04-14 09:45:40.18805 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.18805 +0000 +0000   0  NULL  NULL
        	            	856539595500290049  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-14 09:45:40.228398 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:40.228398 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.228398 +0000 +0000  0  NULL  NULL
        	            	856539595640995841  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-14 09:45:40.271334 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:40.271334 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.271334 +0000 +0000  0  NULL  NULL
        	            	856539595772690433  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-14 09:45:40.311541 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:40.311541 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.311541 +0000 +0000  0  NULL  NULL
        	            	856539595896586241  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-14 09:45:40.349336 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:40.349336 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.349336 +0000 +0000  0  NULL  NULL
        	            	856539596032737281  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-14 09:45:40.390889 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:40.390889 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.390889 +0000 +0000  0  NULL  NULL
        	            	856539596175376385  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-14 09:45:40.434412 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:40.434412 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.434412 +0000 +0000  0  NULL  NULL
        	            	856539596300943361  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-14 09:45:40.472747 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:40.472747 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.472747 +0000 +0000  0  NULL  NULL
        	            	856539596432277505  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-14 09:45:40.512813 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:40.512813 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.512813 +0000 +0000  0  NULL  NULL
        	            	856539596570525697  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-14 09:45:40.555012 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:40.555012 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.555012 +0000 +0000  0  NULL  NULL
        	            	856539596698615809  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-14 09:45:40.594104 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:40.594104 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.594104 +0000 +0000  0  NULL  NULL
        	            	856539596732203009  AUTO CREATE STATS                Table statistics refresh for system.public.reports_meta                                 CREATE STATISTICS __auto__ FROM [28] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-14 09:45:40.604324 +0000 +0000  2023-04-14 09:45:40.712657 +0000 +0000  2023-04-14 09:45:42.289859 +0000 +0000  2023-04-14 09:45:42.289954 +0000 +0000  1  NULL    1     1185155793117370749  2023-04-14 09:45:40.712662 +0000 +0000  2023-04-15 09:45:40.712662 +0000 +0000  1  NULL  NULL
        	            	856539596840796161  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:45:40.637485 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:40.637485 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.637485 +0000 +0000  0  NULL  NULL
        	            	856539596974882817  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:45:40.678408 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:40.678408 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.678408 +0000 +0000  0  NULL  NULL
        	            	856539597122666497  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:45:40.723503 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:40.723503 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.723503 +0000 +0000  0  NULL  NULL
        	            	856539597261799425  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:45:40.765978 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:44.465616 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.765978 +0000 +0000  0  NULL  NULL
        	            	856539597424787457  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:45:40.815712 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:44.543448 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.815712 +0000 +0000  0  NULL  NULL
        	            	856539597561659393  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:45:40.857483 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:44.612087 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.857483 +0000 +0000  0  NULL  NULL
        	            	856539597717929985  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:45:40.905165 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:45.38386 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.905165 +0000 +0000  0  NULL  NULL
        	            	856539597840646145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:45:40.94261 +0000 +0000   NULL                                    NULL                                    2023-04-14 09:45:45.451752 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.94261 +0000 +0000   0  NULL  NULL
        	            	856539597962641409  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:45:40.97984 +0000 +0000   NULL                                    NULL                                    2023-04-14 09:45:45.524545 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:40.97984 +0000 +0000   0  NULL  NULL
        	            	856539598100201473  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-14 09:45:41.021832 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:41.021832 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.021832 +0000 +0000  0  NULL  NULL
        	            	856539598224916481  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-14 09:45:41.059894 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:41.059894 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.059894 +0000 +0000  0  NULL  NULL
        	            	856539598402551809  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-14 09:45:41.114102 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:41.114102 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.114102 +0000 +0000  0  NULL  NULL
        	            	856539598532640769  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-14 09:45:41.153816 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:41.153816 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.153816 +0000 +0000  0  NULL  NULL
        	            	856539598664663041  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-14 09:45:41.194091 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:41.194091 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.194091 +0000 +0000  0  NULL  NULL
        	            	856539598794555393  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-14 09:45:41.233741 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:41.233741 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.233741 +0000 +0000  0  NULL  NULL
        	            	856539599105392641  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:45:41.328666 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:45.148085 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.328666 +0000 +0000  0  NULL  NULL
        	            	856539599248195585  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:45:41.372177 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:45.222546 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.372177 +0000 +0000  0  NULL  NULL
        	            	856539599422062593  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:45:41.425224 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:45.305299 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.425224 +0000 +0000  0  NULL  NULL
        	            	856539599574794241  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-14 09:45:41.471838 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:41.471838 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.471838 +0000 +0000  0  NULL  NULL
        	            	856539599718809601  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-14 09:45:41.51579 +0000 +0000   NULL                                    NULL                                    2023-04-14 09:45:41.51579 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.51579 +0000 +0000   0  NULL  NULL
        	            	856539599868035073  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-14 09:45:41.561339 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:41.561339 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.561339 +0000 +0000  0  NULL  NULL
        	            	856539600029450241  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-14 09:45:41.61061 +0000 +0000   NULL                                    NULL                                    2023-04-14 09:45:41.61061 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.61061 +0000 +0000   0  NULL  NULL
        	            	856539600175529985  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-14 09:45:41.655172 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:41.655172 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.655172 +0000 +0000  0  NULL  NULL
        	            	856539600311123969  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-14 09:45:41.696552 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:41.696552 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.696552 +0000 +0000  0  NULL  NULL
        	            	856539600439083009  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-14 09:45:41.735602 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:41.735602 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.735602 +0000 +0000  0  NULL  NULL
        	            	856539600576937985  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-14 09:45:41.777692 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:41.777692 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.777692 +0000 +0000  0  NULL  NULL
        	            	856539600712531969  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-14 09:45:41.819039 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:41.819039 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.819039 +0000 +0000  0  NULL  NULL
        	            	856539600848289793  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:45:41.860483 +0000 +0000  NULL                                    NULL                                    2023-04-14 09:45:41.860483 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.860483 +0000 +0000  0  NULL  NULL
        	            	856539600987881473  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:45:41.9031 +0000 +0000    NULL                                    NULL                                    2023-04-14 09:45:41.9031 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.9031 +0000 +0000    0  NULL  NULL
        	            	856539601119281153  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-14 09:45:41.9432 +0000 +0000    NULL                                    NULL                                    2023-04-14 09:45:41.9432 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-14 09:45:41.9432 +0000 +0000    0  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (9.80s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ 3c51a373e8c272c5fd3fef0179321df51cbf401e:

        	            	856833370132873217  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-15 10:39:53.131414 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:53.131414 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:53.131414 +0000 +0000  0  NULL  NULL
        	            	856833370369097729  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-15 10:39:53.203487 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:53.203487 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:53.203487 +0000 +0000  0  NULL  NULL
        	            	856833370593460225  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-15 10:39:53.27196 +0000 +0000   NULL                                    NULL                                    2023-04-15 10:39:53.27196 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:53.27196 +0000 +0000   0  NULL  NULL
        	            	856833370833518593  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-15 10:39:53.345227 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:53.345227 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:53.345227 +0000 +0000  0  NULL  NULL
        	            	856833371064270849  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-15 10:39:53.415632 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:53.415632 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:53.415632 +0000 +0000  0  NULL  NULL
        	            	856833371301052417  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-15 10:39:53.487898 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:53.487898 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:53.487898 +0000 +0000  0  NULL  NULL
        	            	856833371524005889  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-15 10:39:53.555944 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:53.555944 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:53.555944 +0000 +0000  0  NULL  NULL
        	            	856833371786903553  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-15 10:39:53.636178 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:53.636178 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:53.636178 +0000 +0000  0  NULL  NULL
        	            	856833372025847809  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-15 10:39:53.709082 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:53.709082 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:53.709082 +0000 +0000  0  NULL  NULL
        	            	856833372256632833  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-15 10:39:53.779521 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:53.779521 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:53.779521 +0000 +0000  0  NULL  NULL
        	            	856833372513107969  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-15 10:39:53.857793 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:53.857793 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:53.857793 +0000 +0000  0  NULL  NULL
        	            	856833372749758465  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-15 10:39:53.930023 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:53.930023 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:53.930023 +0000 +0000  0  NULL  NULL
        	            	856833372970352641  AUTO CREATE STATS                Table statistics refresh for system.public.namespace                                    CREATE STATISTICS __auto__ FROM [30] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-15 10:39:53.997313 +0000 +0000  2023-04-15 10:39:54.16072 +0000 +0000   2023-04-15 10:39:54.883712 +0000 +0000  2023-04-15 10:39:54.883765 +0000 +0000  1  NULL    1     5131802028477137194  2023-04-15 10:39:54.160737 +0000 +0000  2023-04-16 10:39:54.160737 +0000 +0000  1  NULL  NULL
        	            	856833373037821953  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-15 10:39:54.017922 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:54.017922 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:54.017922 +0000 +0000  0  NULL  NULL
        	            	856833373317398529  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-15 10:39:54.103244 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:54.103244 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:54.103244 +0000 +0000  0  NULL  NULL
        	            	856833373557981185  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-15 10:39:54.176664 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:54.176664 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:54.176664 +0000 +0000  0  NULL  NULL
        	            	856833373787226113  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-15 10:39:54.246625 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:54.246625 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:54.246625 +0000 +0000  0  NULL  NULL
        	            	856833374021779457  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:39:54.318208 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:54.318208 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:54.318208 +0000 +0000  0  NULL  NULL
        	            	856833374264852481  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:39:54.392375 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:54.392375 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:54.392375 +0000 +0000  0  NULL  NULL
        	            	856833374505336833  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:39:54.465771 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:54.465771 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:54.465771 +0000 +0000  0  NULL  NULL
        	            	856833374757617665  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:39:54.542762 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:57.692018 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:54.542762 +0000 +0000  0  NULL  NULL
        	            	856833375014420481  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:39:54.621127 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:57.782584 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:54.621127 +0000 +0000  0  NULL  NULL
        	            	856833375256838145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:39:54.695111 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:57.867944 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:54.695111 +0000 +0000  0  NULL  NULL
        	            	856833375497584641  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:39:54.768576 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:58.273088 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:54.768576 +0000 +0000  0  NULL  NULL
        	            	856833375768936449  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:39:54.851395 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:58.361472 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:54.851395 +0000 +0000  0  NULL  NULL
        	            	856833376001556481  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:39:54.922385 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:58.451909 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:54.922385 +0000 +0000  0  NULL  NULL
        	            	856833376299253761  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-15 10:39:55.013235 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:55.013235 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:55.013235 +0000 +0000  0  NULL  NULL
        	            	856833376533676033  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-15 10:39:55.084776 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:55.084776 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:55.084776 +0000 +0000  0  NULL  NULL
        	            	856833376769409025  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-15 10:39:55.156715 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:55.156715 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:55.156715 +0000 +0000  0  NULL  NULL
        	            	856833377019396097  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-15 10:39:55.232999 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:55.232999 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:55.232999 +0000 +0000  0  NULL  NULL
        	            	856833377042169857  AUTO CREATE STATS                Table statistics refresh for system.public.protected_ts_records                         CREATE STATISTICS __auto__ FROM [32] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-15 10:39:55.239942 +0000 +0000  2023-04-15 10:39:55.396145 +0000 +0000  2023-04-15 10:39:56.150776 +0000 +0000  2023-04-15 10:39:56.150829 +0000 +0000  1  NULL    1     8104519599832888319  2023-04-15 10:39:55.396162 +0000 +0000  2023-04-16 10:39:55.396162 +0000 +0000  1  NULL  NULL
        	            	856833377264730113  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-15 10:39:55.307863 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:55.307863 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:55.307863 +0000 +0000  0  NULL  NULL
        	            	856833377514553345  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-15 10:39:55.384112 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:55.384112 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:55.384112 +0000 +0000  0  NULL  NULL
        	            	856833377752842241  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:39:55.456828 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:58.000204 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:55.456828 +0000 +0000  0  NULL  NULL
        	            	856833377986772993  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:39:55.528225 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:58.085433 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:55.528225 +0000 +0000  0  NULL  NULL
        	            	856833378243641345  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:39:55.606621 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:58.174662 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:55.606621 +0000 +0000  0  NULL  NULL
        	            	856833378481930241  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-15 10:39:55.67933 +0000 +0000   NULL                                    NULL                                    2023-04-15 10:39:55.67933 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:55.67933 +0000 +0000   0  NULL  NULL
        	            	856833378739322881  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-15 10:39:55.757883 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:55.757883 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:55.757883 +0000 +0000  0  NULL  NULL
        	            	856833378988785665  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-15 10:39:55.834015 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:55.834015 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:55.834015 +0000 +0000  0  NULL  NULL
        	            	856833379244507137  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-15 10:39:55.91206 +0000 +0000   NULL                                    NULL                                    2023-04-15 10:39:55.91206 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:55.91206 +0000 +0000   0  NULL  NULL
        	            	856833379473227777  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-15 10:39:55.981851 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:55.981851 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:55.981851 +0000 +0000  0  NULL  NULL
        	            	856833379711549441  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-15 10:39:56.054576 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:56.054576 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:56.054576 +0000 +0000  0  NULL  NULL
        	            	856833379954720769  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-15 10:39:56.128792 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:56.128792 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:56.128792 +0000 +0000  0  NULL  NULL
        	            	856833380271783937  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-15 10:39:56.225555 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:56.225555 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:56.225555 +0000 +0000  0  NULL  NULL
        	            	856833380524032001  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-15 10:39:56.302537 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:56.302537 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:56.302537 +0000 +0000  0  NULL  NULL
        	            	856833380760518657  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:39:56.374701 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:56.374701 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:56.374701 +0000 +0000  0  NULL  NULL
        	            	856833381025480705  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:39:56.455572 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:56.455572 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:56.455572 +0000 +0000  0  NULL  NULL
        	            	856833381267406849  AUTO CREATE STATS                Table statistics refresh for system.public.tenant_usage                                 CREATE STATISTICS __auto__ FROM [45] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-15 10:39:56.529377 +0000 +0000  2023-04-15 10:39:56.688226 +0000 +0000  NULL                                    2023-04-15 10:39:56.631246 +0000 +0000  0  NULL    1     627776334489134769   2023-04-15 10:39:56.688245 +0000 +0000  2023-04-16 10:39:56.688245 +0000 +0000  1  NULL  NULL
        	            	856833381275795457  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-15 10:39:56.531942 +0000 +0000  NULL                                    NULL                                    2023-04-15 10:39:56.531942 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-15 10:39:56.531942 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (10.42s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ d9427da31576a61b9390d86258fd7ad4c35c88d5:

        	            	857102993077436417  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-16 09:31:15.524151 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:15.524151 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:15.524151 +0000 +0000  0  NULL  NULL
        	            	857102993319591937  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:31:15.598047 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:15.598047 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:15.598047 +0000 +0000  0  NULL  NULL
        	            	857102993579474945  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:31:15.677387 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:15.677387 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:15.677387 +0000 +0000  0  NULL  NULL
        	            	857102993827889153  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:31:15.75318 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:31:15.75318 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:15.75318 +0000 +0000   0  NULL  NULL
        	            	857102994055659521  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:31:15.822682 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:15.822682 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:15.822682 +0000 +0000  0  NULL  NULL
        	            	857102994291785729  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:31:15.89474 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:31:15.89474 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:15.89474 +0000 +0000   0  NULL  NULL
        	            	857102994534793217  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:31:15.968901 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:15.968901 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:15.968901 +0000 +0000  0  NULL  NULL
        	            	857102994776293377  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:31:16.042598 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:16.042598 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:16.042598 +0000 +0000  0  NULL  NULL
        	            	857102995001901057  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:31:16.111451 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:16.111451 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:16.111451 +0000 +0000  0  NULL  NULL
        	            	857102995229343745  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:31:16.180853 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:16.180853 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:16.180853 +0000 +0000  0  NULL  NULL
        	            	857102995459604481  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:31:16.251142 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:16.251142 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:16.251142 +0000 +0000  0  NULL  NULL
        	            	857102995515211777  AUTO CREATE STATS                Table statistics refresh for system.public.namespace                                    CREATE STATISTICS __auto__ FROM [30] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-16 09:31:16.268089 +0000 +0000  2023-04-16 09:31:16.444014 +0000 +0000  2023-04-16 09:31:17.160944 +0000 +0000  2023-04-16 09:31:17.160998 +0000 +0000  1  NULL    1     8873450994167914466  2023-04-16 09:31:16.444024 +0000 +0000  2023-04-17 09:31:16.444024 +0000 +0000  1  NULL  NULL
        	            	857102995730792449  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:31:16.333887 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:16.333887 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:16.333887 +0000 +0000  0  NULL  NULL
        	            	857102996020264961  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:31:16.422247 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:16.422247 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:16.422247 +0000 +0000  0  NULL  NULL
        	            	857102996259536897  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:31:16.495247 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:16.495247 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:16.495247 +0000 +0000  0  NULL  NULL
        	            	857102996509589505  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:31:16.571557 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:16.571557 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:16.571557 +0000 +0000  0  NULL  NULL
        	            	857102996745945089  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:31:16.64369 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:31:16.64369 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:16.64369 +0000 +0000   0  NULL  NULL
        	            	857102996987052033  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:31:16.717275 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:16.717275 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:16.717275 +0000 +0000  0  NULL  NULL
        	            	857102997234188289  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:31:16.792681 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:16.792681 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:16.792681 +0000 +0000  0  NULL  NULL
        	            	857102997483585537  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:31:16.868805 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:16.868805 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:16.868805 +0000 +0000  0  NULL  NULL
        	            	857102997720465409  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:31:16.941088 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:20.117312 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:16.941088 +0000 +0000  0  NULL  NULL
        	            	857102997957574657  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:31:17.013454 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:20.197543 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:17.013454 +0000 +0000  0  NULL  NULL
        	            	857102998191931393  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:31:17.084963 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:20.27558 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:17.084963 +0000 +0000  0  NULL  NULL
        	            	857102998429433857  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:31:17.157447 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:20.661882 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:17.157447 +0000 +0000  0  NULL  NULL
        	            	857102998661726209  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:31:17.228343 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:20.748946 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:17.228343 +0000 +0000  0  NULL  NULL
        	            	857102998890315777  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:31:17.298108 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:20.842058 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:17.298108 +0000 +0000  0  NULL  NULL
        	            	857102999122935809  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-16 09:31:17.36909 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:31:17.36909 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:17.36909 +0000 +0000   0  NULL  NULL
        	            	857102999369875457  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-16 09:31:17.444449 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:17.444449 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:17.444449 +0000 +0000  0  NULL  NULL
        	            	857102999625367553  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-16 09:31:17.52243 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:31:17.52243 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:17.52243 +0000 +0000   0  NULL  NULL
        	            	857102999784980481  AUTO CREATE STATS                Table statistics refresh for system.public.sql_instances                                CREATE STATISTICS __auto__ FROM [46] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-16 09:31:17.571114 +0000 +0000  2023-04-16 09:31:17.731591 +0000 +0000  2023-04-16 09:31:18.534459 +0000 +0000  2023-04-16 09:31:18.534513 +0000 +0000  1  NULL    1     6598719507838900663  2023-04-16 09:31:17.731609 +0000 +0000  2023-04-17 09:31:17.731609 +0000 +0000  1  NULL  NULL
        	            	857102999925620737  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:31:17.614046 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:17.614046 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:17.614046 +0000 +0000  0  NULL  NULL
        	            	857103000226267137  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:31:17.705804 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:17.705804 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:17.705804 +0000 +0000  0  NULL  NULL
        	            	857103000479956993  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:31:17.783218 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:17.783218 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:17.783218 +0000 +0000  0  NULL  NULL
        	            	857103000726208513  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:31:17.858372 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:20.409525 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:17.858372 +0000 +0000  0  NULL  NULL
        	            	857103000988975105  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:31:17.938587 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:20.493228 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:17.938587 +0000 +0000  0  NULL  NULL
        	            	857103001264259073  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:31:18.022573 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:20.57492 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:18.022573 +0000 +0000  0  NULL  NULL
        	            	857103001509101569  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:31:18.097287 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:18.097287 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:18.097287 +0000 +0000  0  NULL  NULL
        	            	857103001744998401  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:31:18.169287 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:18.169287 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:18.169287 +0000 +0000  0  NULL  NULL
        	            	857103001974013953  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:31:18.239176 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:18.239176 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:18.239176 +0000 +0000  0  NULL  NULL
        	            	857103002213842945  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:31:18.312359 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:18.312359 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:18.312359 +0000 +0000  0  NULL  NULL
        	            	857103002442825729  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:31:18.382239 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:18.382239 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:18.382239 +0000 +0000  0  NULL  NULL
        	            	857103002667876353  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:31:18.450921 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:18.450921 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:18.450921 +0000 +0000  0  NULL  NULL
        	            	857103002907082753  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:31:18.523991 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:18.523991 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:18.523991 +0000 +0000  0  NULL  NULL
        	            	857103003147436033  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:31:18.597268 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:18.597268 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:18.597268 +0000 +0000  0  NULL  NULL
        	            	857103003386970113  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:31:18.670389 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:18.670389 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:18.670389 +0000 +0000  0  NULL  NULL
        	            	857103003620081665  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:31:18.74151 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:31:18.74151 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:18.74151 +0000 +0000   0  NULL  NULL
        	            	857103003906179073  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:31:18.828844 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:18.828844 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:18.828844 +0000 +0000  0  NULL  NULL
        	            	857103004159377409  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:31:18.906096 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:31:18.906096 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:31:18.906096 +0000 +0000  0  NULL  NULL
        	            	857103004298641409  AUTO CREATE STATS                Table statistics refresh for system.public.tenant_tasks                                 CREATE STATISTICS __auto__ FROM [59] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-16 09:31:18.948574 +0000 +0000  2023-04-16 09:31:19.091243 +0000 +0000  NULL                                    2023-04-16 09:31:19.040792 +0000 +0000  0  NULL    1     3364477811241506625  2023-04-16 09:31:19.091269 +0000 +0000  2023-04-17 09:31:19.091269 +0000 +0000  1  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (10.43s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ d9427da31576a61b9390d86258fd7ad4c35c88d5:

        	            	857103289511510017  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-16 09:32:45.98865 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:32:45.98865 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:45.98865 +0000 +0000   0  NULL  NULL
        	            	857103289649659905  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-16 09:32:46.030818 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:46.030818 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.030818 +0000 +0000  0  NULL  NULL
        	            	857103289788399617  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-16 09:32:46.073155 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:46.073155 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.073155 +0000 +0000  0  NULL  NULL
        	            	857103289931661313  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:32:46.116876 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:46.116876 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.116876 +0000 +0000  0  NULL  NULL
        	            	857103290078789633  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:32:46.161781 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:46.161781 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.161781 +0000 +0000  0  NULL  NULL
        	            	857103290220052481  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:32:46.204882 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:46.204882 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.204882 +0000 +0000  0  NULL  NULL
        	            	857103290366820353  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:32:46.249671 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:46.249671 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.249671 +0000 +0000  0  NULL  NULL
        	            	857103290517258241  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:32:46.295592 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:46.295592 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.295592 +0000 +0000  0  NULL  NULL
        	            	857103290688536577  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:32:46.347865 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:46.347865 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.347865 +0000 +0000  0  NULL  NULL
        	            	857103290828881921  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:32:46.39069 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:32:46.39069 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.39069 +0000 +0000   0  NULL  NULL
        	            	857103290970963969  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:32:46.434045 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:46.434045 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.434045 +0000 +0000  0  NULL  NULL
        	            	857103291114749953  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:32:46.47792 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:32:46.47792 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.47792 +0000 +0000   0  NULL  NULL
        	            	857103291291828225  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:32:46.531968 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:46.531968 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.531968 +0000 +0000  0  NULL  NULL
        	            	857103291454488577  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:32:46.581599 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:46.581599 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.581599 +0000 +0000  0  NULL  NULL
        	            	857103291599421441  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:32:46.625831 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:46.625831 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.625831 +0000 +0000  0  NULL  NULL
        	            	857103291800223745  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:32:46.687118 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:46.687118 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.687118 +0000 +0000  0  NULL  NULL
        	            	857103291946631169  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:32:46.731813 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:46.731813 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.731813 +0000 +0000  0  NULL  NULL
        	            	857103292097789953  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:32:46.777929 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:46.777929 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.777929 +0000 +0000  0  NULL  NULL
        	            	857103292284731393  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:32:46.834977 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:46.834977 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.834977 +0000 +0000  0  NULL  NULL
        	            	857103292436971521  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:32:46.881439 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:46.881439 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.881439 +0000 +0000  0  NULL  NULL
        	            	857103292585213953  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:32:46.926678 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:46.926678 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.926678 +0000 +0000  0  NULL  NULL
        	            	857103292744040449  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-16 09:32:46.97515 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:32:46.97515 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:46.97515 +0000 +0000   0  NULL  NULL
        	            	857103292884942849  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-16 09:32:47.018156 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.018156 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.018156 +0000 +0000  0  NULL  NULL
        	            	857103293024403457  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-16 09:32:47.060714 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.060714 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.060714 +0000 +0000  0  NULL  NULL
        	            	857103293167075329  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-16 09:32:47.104265 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.104265 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.104265 +0000 +0000  0  NULL  NULL
        	            	857103293301850113  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-16 09:32:47.145382 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.145382 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.145382 +0000 +0000  0  NULL  NULL
        	            	857103293437116417  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-16 09:32:47.186659 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.186659 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.186659 +0000 +0000  0  NULL  NULL
        	            	857103293588963329  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-16 09:32:47.233008 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.233008 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.233008 +0000 +0000  0  NULL  NULL
        	            	857103293732782081  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-16 09:32:47.276908 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.276908 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.276908 +0000 +0000  0  NULL  NULL
        	            	857103293919854593  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-16 09:32:47.333983 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.333983 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.333983 +0000 +0000  0  NULL  NULL
        	            	857103294086742017  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:32:47.384937 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.384937 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.384937 +0000 +0000  0  NULL  NULL
        	            	857103294227218433  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:32:47.427802 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.427802 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.427802 +0000 +0000  0  NULL  NULL
        	            	857103294368841729  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-16 09:32:47.470998 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.470998 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.470998 +0000 +0000  0  NULL  NULL
        	            	857103294508892161  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:32:47.513741 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.513741 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.513741 +0000 +0000  0  NULL  NULL
        	            	857103294648254465  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:32:47.556271 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.556271 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.556271 +0000 +0000  0  NULL  NULL
        	            	857103294801772545  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-16 09:32:47.603125 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.603125 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.603125 +0000 +0000  0  NULL  NULL
        	            	857103294944575489  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:32:47.646699 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.646699 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.646699 +0000 +0000  0  NULL  NULL
        	            	857103295088885761  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:32:47.690747 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.690747 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.690747 +0000 +0000  0  NULL  NULL
        	            	857103295228215297  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-16 09:32:47.733262 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.733262 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.733262 +0000 +0000  0  NULL  NULL
        	            	857103295373410305  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:32:47.777569 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.777569 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.777569 +0000 +0000  0  NULL  NULL
        	            	857103295515328513  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:32:47.82088 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:32:47.82088 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.82088 +0000 +0000   0  NULL  NULL
        	            	857103295654428673  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-16 09:32:47.863332 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.863332 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.863332 +0000 +0000  0  NULL  NULL
        	            	857103295798607873  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:32:47.907327 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.907327 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.907327 +0000 +0000  0  NULL  NULL
        	            	857103295952977921  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:32:47.954441 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.954441 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.954441 +0000 +0000  0  NULL  NULL
        	            	857103296098271233  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-16 09:32:47.998792 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:47.998792 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:47.998792 +0000 +0000  0  NULL  NULL
        	            	857103296257949697  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:32:48.047508 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:48.047508 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:48.047508 +0000 +0000  0  NULL  NULL
        	            	857103296419921921  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:32:48.09696 +0000 +0000   NULL                                    NULL                                    2023-04-16 09:32:48.09696 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:48.09696 +0000 +0000   0  NULL  NULL
        	            	857103296480411649  AUTO CREATE STATS                Table statistics refresh for system.public.descriptor_id_seq                            CREATE STATISTICS __auto__ FROM [7] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'   root      {}  running           NULL  2023-04-16 09:32:48.115377 +0000 +0000  2023-04-16 09:32:48.229061 +0000 +0000  NULL                                    2023-04-16 09:32:48.186834 +0000 +0000  0  NULL    1     5949035389986028611  2023-04-16 09:32:48.229066 +0000 +0000  2023-04-17 09:32:48.229066 +0000 +0000  1  NULL  NULL
        	            	857103296571179009  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-16 09:32:48.143101 +0000 +0000  NULL                                    NULL                                    2023-04-16 09:32:48.143101 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-16 09:32:48.143101 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (7.99s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ d9427da31576a61b9390d86258fd7ad4c35c88d5:

        	            	857408387878322177  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-17 11:24:34.621112 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:34.621112 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:34.621112 +0000 +0000  0  NULL  NULL
        	            	857408388052484097  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-17 11:24:34.674255 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:34.674255 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:34.674255 +0000 +0000  0  NULL  NULL
        	            	857408388214423553  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-17 11:24:34.72369 +0000 +0000   NULL                                    NULL                                    2023-04-17 11:24:34.72369 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:34.72369 +0000 +0000   0  NULL  NULL
        	            	857408388377378817  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 11:24:34.773406 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:34.773406 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:34.773406 +0000 +0000  0  NULL  NULL
        	            	857408388539580417  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 11:24:34.822911 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:34.822911 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:34.822911 +0000 +0000  0  NULL  NULL
        	            	857408388695556097  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 11:24:34.870514 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:34.870514 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:34.870514 +0000 +0000  0  NULL  NULL
        	            	857408388832034817  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 11:24:34.912153 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:34.912153 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:34.912153 +0000 +0000  0  NULL  NULL
        	            	857408388968710145  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 11:24:34.95386 +0000 +0000   NULL                                    NULL                                    2023-04-17 11:24:34.95386 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:34.95386 +0000 +0000   0  NULL  NULL
        	            	857408389101322241  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 11:24:34.994341 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:34.994341 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:34.994341 +0000 +0000  0  NULL  NULL
        	            	857408389233967105  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 11:24:35.034823 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:35.034823 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.034823 +0000 +0000  0  NULL  NULL
        	            	857408389370183681  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 11:24:35.0764 +0000 +0000    NULL                                    NULL                                    2023-04-17 11:24:35.0764 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.0764 +0000 +0000    0  NULL  NULL
        	            	857408389507153921  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 11:24:35.118186 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:35.118186 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.118186 +0000 +0000  0  NULL  NULL
        	            	857408389652611073  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 11:24:35.162583 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:35.162583 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.162583 +0000 +0000  0  NULL  NULL
        	            	857408389792858113  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 11:24:35.205389 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:35.205389 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.205389 +0000 +0000  0  NULL  NULL
        	            	857408389932417025  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 11:24:35.247961 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:35.247961 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.247961 +0000 +0000  0  NULL  NULL
        	            	857408390100189185  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 11:24:35.29917 +0000 +0000   NULL                                    NULL                                    2023-04-17 11:24:35.29917 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.29917 +0000 +0000   0  NULL  NULL
        	            	857408390241517569  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 11:24:35.342306 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:35.342306 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.342306 +0000 +0000  0  NULL  NULL
        	            	857408390417186817  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 11:24:35.39591 +0000 +0000   NULL                                    NULL                                    2023-04-17 11:24:35.39591 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.39591 +0000 +0000   0  NULL  NULL
        	            	857408390574178305  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:24:35.44381 +0000 +0000   NULL                                    NULL                                    2023-04-17 11:24:35.44381 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.44381 +0000 +0000   0  NULL  NULL
        	            	857408390728712193  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:24:35.490971 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:35.490971 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.490971 +0000 +0000  0  NULL  NULL
        	            	857408390878232577  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:24:35.536608 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:35.536608 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.536608 +0000 +0000  0  NULL  NULL
        	            	857408391045644289  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:24:35.587705 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:39.422539 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.587705 +0000 +0000  0  NULL  NULL
        	            	857408391243399169  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:24:35.648058 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:39.498161 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.648058 +0000 +0000  0  NULL  NULL
        	            	857408391447183361  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:24:35.710237 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:39.574214 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.710237 +0000 +0000  0  NULL  NULL
        	            	857408391627440129  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:24:35.765249 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:40.309509 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.765249 +0000 +0000  0  NULL  NULL
        	            	857408391779221505  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:24:35.811575 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:40.381184 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.811575 +0000 +0000  0  NULL  NULL
        	            	857408391910588417  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:24:35.851668 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:40.45824 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.851668 +0000 +0000  0  NULL  NULL
        	            	857408392044806145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-17 11:24:35.892618 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:35.892618 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.892618 +0000 +0000  0  NULL  NULL
        	            	857408392186888193  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-17 11:24:35.935981 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:35.935981 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.935981 +0000 +0000  0  NULL  NULL
        	            	857408392379105281  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-17 11:24:35.994667 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:35.994667 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:35.994667 +0000 +0000  0  NULL  NULL
        	            	857408392536555521  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 11:24:36.042695 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:36.042695 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:36.042695 +0000 +0000  0  NULL  NULL
        	            	857408392685092865  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 11:24:36.088013 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:36.088013 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:36.088013 +0000 +0000  0  NULL  NULL
        	            	857408392818327553  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 11:24:36.128681 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:36.128681 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:36.128681 +0000 +0000  0  NULL  NULL
        	            	857408392947499009  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:24:36.168099 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:40.084507 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:36.168099 +0000 +0000  0  NULL  NULL
        	            	857408393078767617  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:24:36.208168 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:40.158176 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:36.208168 +0000 +0000  0  NULL  NULL
        	            	857408393214918657  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:24:36.249703 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:40.227549 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:36.249703 +0000 +0000  0  NULL  NULL
        	            	857408393358082049  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 11:24:36.293402 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:36.293402 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:36.293402 +0000 +0000  0  NULL  NULL
        	            	857408393505800193  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 11:24:36.33848 +0000 +0000   NULL                                    NULL                                    2023-04-17 11:24:36.33848 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:36.33848 +0000 +0000   0  NULL  NULL
        	            	857408393647194113  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 11:24:36.381635 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:36.381635 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:36.381635 +0000 +0000  0  NULL  NULL
        	            	857408393786294273  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 11:24:36.424079 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:36.424079 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:36.424079 +0000 +0000  0  NULL  NULL
        	            	857408393917202433  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 11:24:36.464025 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:36.464025 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:36.464025 +0000 +0000  0  NULL  NULL
        	            	857408394054107137  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 11:24:36.505816 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:36.505816 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:36.505816 +0000 +0000  0  NULL  NULL
        	            	857408394200580097  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 11:24:36.55051 +0000 +0000   NULL                                    NULL                                    2023-04-17 11:24:36.55051 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:36.55051 +0000 +0000   0  NULL  NULL
        	            	857408394334339073  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 11:24:36.591333 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:36.591333 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:36.591333 +0000 +0000  0  NULL  NULL
        	            	857408394481696769  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 11:24:36.636302 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:36.636302 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:36.636302 +0000 +0000  0  NULL  NULL
        	            	857408394652352513  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:24:36.688387 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:36.688387 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:36.688387 +0000 +0000  0  NULL  NULL
        	            	857408394804461569  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:24:36.734808 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:36.734808 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:36.734808 +0000 +0000  0  NULL  NULL
        	            	857408394955718657  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:24:36.781007 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:24:36.781007 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:24:36.781007 +0000 +0000  0  NULL  NULL
        	            	857408395684839425  AUTO CREATE STATS                Table statistics refresh for system.public.comments                                     CREATE STATISTICS __auto__ FROM [24] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-17 11:24:37.003443 +0000 +0000  2023-04-17 11:24:37.127608 +0000 +0000  NULL                                    2023-04-17 11:24:37.082427 +0000 +0000  0  NULL    1     8906417696710301705  2023-04-17 11:24:37.127614 +0000 +0000  2023-04-18 11:24:37.127614 +0000 +0000  1  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (9.94s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ d9427da31576a61b9390d86258fd7ad4c35c88d5:

        	            	857411282062114817  BACKUP                           fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-17 11:39:17.855899 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:17.855899 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:17.855899 +0000 +0000  0  NULL  NULL
        	            	857411282277269505  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 11:39:17.921563 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:17.921563 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:17.921563 +0000 +0000  0  NULL  NULL
        	            	857411282507497473  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 11:39:17.991825 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:17.991825 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:17.991825 +0000 +0000  0  NULL  NULL
        	            	857411282726486017  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 11:39:18.058667 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:18.058667 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:18.058667 +0000 +0000  0  NULL  NULL
        	            	857411282944360449  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 11:39:18.125138 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:18.125138 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:18.125138 +0000 +0000  0  NULL  NULL
        	            	857411283158269953  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 11:39:18.190435 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:18.190435 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:18.190435 +0000 +0000  0  NULL  NULL
        	            	857411283388760065  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 11:39:18.260762 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:18.260762 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:18.260762 +0000 +0000  0  NULL  NULL
        	            	857411283606601729  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 11:39:18.327238 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:18.327238 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:18.327238 +0000 +0000  0  NULL  NULL
        	            	857411283818446849  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 11:39:18.391887 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:18.391887 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:18.391887 +0000 +0000  0  NULL  NULL
        	            	857411284033568769  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 11:39:18.457545 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:18.457545 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:18.457545 +0000 +0000  0  NULL  NULL
        	            	857411284252753921  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 11:39:18.524425 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:18.524425 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:18.524425 +0000 +0000  0  NULL  NULL
        	            	857411284547076097  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 11:39:18.614255 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:18.614255 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:18.614255 +0000 +0000  0  NULL  NULL
        	            	857411284792639489  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 11:39:18.689189 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:18.689189 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:18.689189 +0000 +0000  0  NULL  NULL
        	            	857411285042462721  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 11:39:18.765432 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:18.765432 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:18.765432 +0000 +0000  0  NULL  NULL
        	            	857411285280915457  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 11:39:18.838199 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:18.838199 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:18.838199 +0000 +0000  0  NULL  NULL
        	            	857411285465595905  AUTO CREATE STATS                Table statistics refresh for system.public.namespace                                    CREATE STATISTICS __auto__ FROM [30] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-17 11:39:18.894539 +0000 +0000  2023-04-17 11:39:19.047057 +0000 +0000  2023-04-17 11:39:19.76348 +0000 +0000   2023-04-17 11:39:19.763528 +0000 +0000  1  NULL    1     4525665143108727444  2023-04-17 11:39:19.04707 +0000 +0000   2023-04-18 11:39:19.04707 +0000 +0000   1  NULL  NULL
        	            	857411285509177345  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 11:39:18.907851 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:18.907851 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:18.907851 +0000 +0000  0  NULL  NULL
        	            	857411285731704833  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:39:18.975761 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:18.975761 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:18.975761 +0000 +0000  0  NULL  NULL
        	            	857411286061121537  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:39:19.07632 +0000 +0000   NULL                                    NULL                                    2023-04-17 11:39:19.07632 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:19.07632 +0000 +0000   0  NULL  NULL
        	            	857411286269919233  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:39:19.140035 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:19.140035 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:19.140035 +0000 +0000  0  NULL  NULL
        	            	857411286502834177  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-17 11:39:19.211098 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:19.211098 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:19.211098 +0000 +0000  0  NULL  NULL
        	            	857411286810165249  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-17 11:39:19.30488 +0000 +0000   NULL                                    NULL                                    2023-04-17 11:39:19.30488 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:19.30488 +0000 +0000   0  NULL  NULL
        	            	857411287057530881  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-17 11:39:19.380381 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:19.380381 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:19.380381 +0000 +0000  0  NULL  NULL
        	            	857411287283728385  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-17 11:39:19.449409 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:19.449409 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:19.449409 +0000 +0000  0  NULL  NULL
        	            	857411287508189185  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-17 11:39:19.517902 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:19.517902 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:19.517902 +0000 +0000  0  NULL  NULL
        	            	857411287743692801  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-17 11:39:19.589778 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:19.589778 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:19.589778 +0000 +0000  0  NULL  NULL
        	            	857411287980802049  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-17 11:39:19.662134 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:19.662134 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:19.662134 +0000 +0000  0  NULL  NULL
        	            	857411288216633345  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-17 11:39:19.734106 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:19.734106 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:19.734106 +0000 +0000  0  NULL  NULL
        	            	857411288450793473  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-17 11:39:19.805565 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:19.805565 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:19.805565 +0000 +0000  0  NULL  NULL
        	            	857411288692850689  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 11:39:19.879438 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:19.879438 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:19.879438 +0000 +0000  0  NULL  NULL
        	            	857411288962695169  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 11:39:19.961795 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:19.961795 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:19.961795 +0000 +0000  0  NULL  NULL
        	            	857411289206456321  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-17 11:39:20.03618 +0000 +0000   NULL                                    NULL                                    2023-04-17 11:39:20.03618 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:20.03618 +0000 +0000   0  NULL  NULL
        	            	857411289429475329  AUTO CREATE STATS                Table statistics refresh for system.public.span_stats_samples                           CREATE STATISTICS __auto__ FROM [56] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-17 11:39:20.104232 +0000 +0000  2023-04-17 11:39:20.257156 +0000 +0000  2023-04-17 11:39:20.797052 +0000 +0000  2023-04-17 11:39:20.7971 +0000 +0000    1  NULL    1     5505836129632572866  2023-04-17 11:39:20.257172 +0000 +0000  2023-04-18 11:39:20.257172 +0000 +0000  1  NULL  NULL
        	            	857411289473941505  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 11:39:20.117808 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:20.117808 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:20.117808 +0000 +0000  0  NULL  NULL
        	            	857411289705480193  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 11:39:20.188476 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:20.188476 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:20.188476 +0000 +0000  0  NULL  NULL
        	            	857411289950519297  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-17 11:39:20.263249 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:20.263249 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:20.263249 +0000 +0000  0  NULL  NULL
        	            	857411290187268097  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 11:39:20.335492 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:20.335492 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:20.335492 +0000 +0000  0  NULL  NULL
        	            	857411290413105153  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 11:39:20.404424 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:20.404424 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:20.404424 +0000 +0000  0  NULL  NULL
        	            	857411290691174401  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-17 11:39:20.489275 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:20.489275 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:20.489275 +0000 +0000  0  NULL  NULL
        	            	857411290914652161  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 11:39:20.557476 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:20.557476 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:20.557476 +0000 +0000  0  NULL  NULL
        	            	857411291205140481  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 11:39:20.646222 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:20.646222 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:20.646222 +0000 +0000  0  NULL  NULL
        	            	857411291433500673  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-17 11:39:20.715822 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:20.715822 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:20.715822 +0000 +0000  0  NULL  NULL
        	            	857411291670446081  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 11:39:20.788129 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:20.788129 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:20.788129 +0000 +0000  0  NULL  NULL
        	            	857411291903787009  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 11:39:20.859341 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:20.859341 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:20.859341 +0000 +0000  0  NULL  NULL
        	            	857411292141158401  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-17 11:39:20.931784 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:20.931784 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:20.931784 +0000 +0000  0  NULL  NULL
        	            	857411292394127361  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:39:21.008984 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:21.008984 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:21.008984 +0000 +0000  0  NULL  NULL
        	            	857411292625371137  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:39:21.079558 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:21.079558 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:21.079558 +0000 +0000  0  NULL  NULL
        	            	857411292827254785  AUTO CREATE STATS                Table statistics refresh for system.public.tenant_tasks                                 CREATE STATISTICS __auto__ FROM [59] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-17 11:39:21.141146 +0000 +0000  2023-04-17 11:39:21.287756 +0000 +0000  NULL                                    2023-04-17 11:39:21.240291 +0000 +0000  0  NULL    1     1086383930194768410  2023-04-17 11:39:21.287778 +0000 +0000  2023-04-18 11:39:21.287778 +0000 +0000  1  NULL  NULL
        	            	857411292909600769  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-17 11:39:21.166283 +0000 +0000  NULL                                    NULL                                    2023-04-17 11:39:21.166283 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-17 11:39:21.166283 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/resume_ALL_BACKUP_JOBS (8.27s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ c7f71068d537c2f82f06f000299144c31d9ddf7a:

        	            	857674168560582657  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 09:56:24.448446 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:24.448446 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:24.448446 +0000 +0000  0  NULL  NULL
        	            	857674168793366529  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 09:56:24.519493 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:24.519493 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:24.519493 +0000 +0000  0  NULL  NULL
        	            	857674169032802305  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 09:56:24.592563 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:24.592563 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:24.592563 +0000 +0000  0  NULL  NULL
        	            	857674169270763521  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 09:56:24.66518 +0000 +0000   NULL                                    NULL                                    2023-04-18 09:56:24.66518 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:24.66518 +0000 +0000   0  NULL  NULL
        	            	857674169495912449  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-18 09:56:24.733889 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:24.733889 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:24.733889 +0000 +0000  0  NULL  NULL
        	            	857674169718013953  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-18 09:56:24.801662 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:24.801662 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:24.801662 +0000 +0000  0  NULL  NULL
        	            	857674169944244225  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-18 09:56:24.870716 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:24.870716 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:24.870716 +0000 +0000  0  NULL  NULL
        	            	857674170166476801  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 09:56:24.938527 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:24.938527 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:24.938527 +0000 +0000  0  NULL  NULL
        	            	857674170180173825  AUTO CREATE STATS                Table statistics refresh for system.public.users                                        CREATE STATISTICS __auto__ FROM [4] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'   root      {}  succeeded         NULL  2023-04-18 09:56:24.942695 +0000 +0000  2023-04-18 09:56:25.098244 +0000 +0000  2023-04-18 09:56:25.765485 +0000 +0000  2023-04-18 09:56:25.765535 +0000 +0000  1  NULL    1     5135733999676096597  2023-04-18 09:56:25.098263 +0000 +0000  2023-04-19 09:56:25.098263 +0000 +0000  1  NULL  NULL
        	            	857674170415775745  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 09:56:25.014663 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:25.014663 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:25.014663 +0000 +0000  0  NULL  NULL
        	            	857674170664845313  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 09:56:25.09062 +0000 +0000   NULL                                    NULL                                    2023-04-18 09:56:25.09062 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:25.09062 +0000 +0000   0  NULL  NULL
        	            	857674170897006593  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 09:56:25.16148 +0000 +0000   NULL                                    NULL                                    2023-04-18 09:56:25.16148 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:25.16148 +0000 +0000   0  NULL  NULL
        	            	857674171131691009  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 09:56:25.233098 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:25.233098 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:25.233098 +0000 +0000  0  NULL  NULL
        	            	857674171400585217  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 09:56:25.315149 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:25.315149 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:25.315149 +0000 +0000  0  NULL  NULL
        	            	857674171643625473  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 09:56:25.389325 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:25.389325 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:25.389325 +0000 +0000  0  NULL  NULL
        	            	857674171916943361  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 09:56:25.472728 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:25.472728 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:25.472728 +0000 +0000  0  NULL  NULL
        	            	857674172159426561  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 09:56:25.546736 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:25.546736 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:25.546736 +0000 +0000  0  NULL  NULL
        	            	857674172429041665  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:56:25.629009 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:25.629009 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:25.629009 +0000 +0000  0  NULL  NULL
        	            	857674172665364481  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:56:25.701132 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:25.701132 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:25.701132 +0000 +0000  0  NULL  NULL
        	            	857674172895952897  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:56:25.771496 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:25.771496 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:25.771496 +0000 +0000  0  NULL  NULL
        	            	857674173139156993  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:56:25.845721 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:29.128877 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:25.845721 +0000 +0000  0  NULL  NULL
        	            	857674173365583873  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:56:25.914819 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:29.212086 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:25.914819 +0000 +0000  0  NULL  NULL
        	            	857674173615472641  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:56:25.991083 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:29.292878 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:25.991083 +0000 +0000  0  NULL  NULL
        	            	857674173850419201  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:56:26.062784 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:29.686319 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:26.062784 +0000 +0000  0  NULL  NULL
        	            	857674174081073153  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:56:26.133193 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:29.768397 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:26.133193 +0000 +0000  0  NULL  NULL
        	            	857674174316675073  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:56:26.205084 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:29.850879 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:26.205084 +0000 +0000  0  NULL  NULL
        	            	857674174383980545  AUTO CREATE STATS                Table statistics refresh for system.public.web_sessions                                 CREATE STATISTICS __auto__ FROM [19] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-18 09:56:26.225595 +0000 +0000  2023-04-18 09:56:26.37578 +0000 +0000   2023-04-18 09:56:27.272857 +0000 +0000  2023-04-18 09:56:27.272918 +0000 +0000  1  NULL    1     8692130595390812380  2023-04-18 09:56:26.375798 +0000 +0000  2023-04-19 09:56:26.375798 +0000 +0000  1  NULL  NULL
        	            	857674174553849857  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 09:56:26.277446 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:26.277446 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:26.277446 +0000 +0000  0  NULL  NULL
        	            	857674174789255169  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 09:56:26.3493 +0000 +0000    NULL                                    NULL                                    2023-04-18 09:56:26.3493 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:26.3493 +0000 +0000    0  NULL  NULL
        	            	857674175029182465  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 09:56:26.42251 +0000 +0000   NULL                                    NULL                                    2023-04-18 09:56:26.42251 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:26.42251 +0000 +0000   0  NULL  NULL
        	            	857674175270354945  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 09:56:26.496109 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:26.496109 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:26.496109 +0000 +0000  0  NULL  NULL
        	            	857674175492685825  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 09:56:26.563962 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:26.563962 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:26.563962 +0000 +0000  0  NULL  NULL
        	            	857674175730581505  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 09:56:26.636564 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:26.636564 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:26.636564 +0000 +0000  0  NULL  NULL
        	            	857674175960743937  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:56:26.70681 +0000 +0000   NULL                                    NULL                                    2023-04-18 09:56:29.433369 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:26.70681 +0000 +0000   0  NULL  NULL
        	            	857674176211320833  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:56:26.783273 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:29.510644 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:26.783273 +0000 +0000  0  NULL  NULL
        	            	857674176458129409  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:56:26.858589 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:29.59215 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:26.858589 +0000 +0000  0  NULL  NULL
        	            	857674176697401345  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 09:56:26.931612 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:26.931612 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:26.931612 +0000 +0000  0  NULL  NULL
        	            	857674176952991745  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 09:56:27.009613 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:27.009613 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:27.009613 +0000 +0000  0  NULL  NULL
        	            	857674177182400513  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 09:56:27.079616 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:27.079616 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:27.079616 +0000 +0000  0  NULL  NULL
        	            	857674177427439617  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 09:56:27.154393 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:27.154393 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:27.154393 +0000 +0000  0  NULL  NULL
        	            	857674177719107585  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 09:56:27.243414 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:27.243414 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:27.243414 +0000 +0000  0  NULL  NULL
        	            	857674178013167617  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 09:56:27.333175 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:27.333175 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:27.333175 +0000 +0000  0  NULL  NULL
        	            	857674178274066433  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 09:56:27.412814 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:27.412814 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:27.412814 +0000 +0000  0  NULL  NULL
        	            	857674178504818689  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 09:56:27.483203 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:27.483203 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:27.483203 +0000 +0000  0  NULL  NULL
        	            	857674178732883969  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 09:56:27.552786 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:27.552786 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:27.552786 +0000 +0000  0  NULL  NULL
        	            	857674178976808961  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:56:27.627232 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:27.627232 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:27.627232 +0000 +0000  0  NULL  NULL
        	            	857674179208511489  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:56:27.697946 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:27.697946 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:27.697946 +0000 +0000  0  NULL  NULL
        	            	857674179290136577  AUTO CREATE STATS                Table statistics refresh for system.public.locations                                    CREATE STATISTICS __auto__ FROM [21] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-18 09:56:27.72283 +0000 +0000   2023-04-18 09:56:27.865774 +0000 +0000  2023-04-18 09:56:28.588463 +0000 +0000  2023-04-18 09:56:28.588539 +0000 +0000  1  NULL    1     3080076619671642413  2023-04-18 09:56:27.865787 +0000 +0000  2023-04-19 09:56:27.865787 +0000 +0000  1  NULL  NULL
        	            	857674179441295361  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:56:27.768974 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:56:27.768974 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:56:27.768974 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (10.46s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ c7f71068d537c2f82f06f000299144c31d9ddf7a:

        	            	857674477900431361  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 09:57:58.851483 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:58.851483 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:58.851483 +0000 +0000  0  NULL  NULL
        	            	857674478039891969  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 09:57:58.894041 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:58.894041 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:58.894041 +0000 +0000  0  NULL  NULL
        	            	857674478180007937  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 09:57:58.936809 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:58.936809 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:58.936809 +0000 +0000  0  NULL  NULL
        	            	857674478321041409  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 09:57:58.979847 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:58.979847 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:58.979847 +0000 +0000  0  NULL  NULL
        	            	857674478461026305  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 09:57:59.02257 +0000 +0000   NULL                                    NULL                                    2023-04-18 09:57:59.02257 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.02257 +0000 +0000   0  NULL  NULL
        	            	857674478600224769  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 09:57:59.065046 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:59.065046 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.065046 +0000 +0000  0  NULL  NULL
        	            	857674478738046977  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-18 09:57:59.107103 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:59.107103 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.107103 +0000 +0000  0  NULL  NULL
        	            	857674478880456705  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-18 09:57:59.150561 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:59.150561 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.150561 +0000 +0000  0  NULL  NULL
        	            	857674479023882241  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-18 09:57:59.194348 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:59.194348 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.194348 +0000 +0000  0  NULL  NULL
        	            	857674479158886401  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 09:57:59.235538 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:59.235538 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.235538 +0000 +0000  0  NULL  NULL
        	            	857674479292907521  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 09:57:59.276443 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:59.276443 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.276443 +0000 +0000  0  NULL  NULL
        	            	857674479450587137  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 09:57:59.324574 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:59.324574 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.324574 +0000 +0000  0  NULL  NULL
        	            	857674479614066689  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 09:57:59.374455 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:59.374455 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.374455 +0000 +0000  0  NULL  NULL
        	            	857674479764144129  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 09:57:59.420241 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:59.420241 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.420241 +0000 +0000  0  NULL  NULL
        	            	857674479907602433  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 09:57:59.464027 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:59.464027 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.464027 +0000 +0000  0  NULL  NULL
        	            	857674480052568065  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 09:57:59.508281 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:59.508281 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.508281 +0000 +0000  0  NULL  NULL
        	            	857674480188293121  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 09:57:59.5497 +0000 +0000    NULL                                    NULL                                    2023-04-18 09:57:59.5497 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.5497 +0000 +0000    0  NULL  NULL
        	            	857674480327327745  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 09:57:59.592126 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:59.592126 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.592126 +0000 +0000  0  NULL  NULL
        	            	857674480473079809  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:57:59.636598 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:59.636598 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.636598 +0000 +0000  0  NULL  NULL
        	            	857674480612868097  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:57:59.679265 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:59.679265 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.679265 +0000 +0000  0  NULL  NULL
        	            	857674480762388481  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:57:59.724887 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:57:59.724887 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.724887 +0000 +0000  0  NULL  NULL
        	            	857674480958144513  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:57:59.784666 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:03.717118 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.784666 +0000 +0000  0  NULL  NULL
        	            	857674481097932801  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:57:59.827281 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:03.786745 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.827281 +0000 +0000  0  NULL  NULL
        	            	857674481267736577  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:57:59.87912 +0000 +0000   NULL                                    NULL                                    2023-04-18 09:58:03.860474 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.87912 +0000 +0000   0  NULL  NULL
        	            	857674481409228801  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:57:59.922301 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:04.692702 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.922301 +0000 +0000  0  NULL  NULL
        	            	857674481545347073  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:57:59.96383 +0000 +0000   NULL                                    NULL                                    2023-04-18 09:58:04.785742 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:57:59.96383 +0000 +0000   0  NULL  NULL
        	            	857674481687166977  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:58:00.007113 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:04.867791 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.007113 +0000 +0000  0  NULL  NULL
        	            	857674481822564353  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 09:58:00.048424 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:00.048424 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.048424 +0000 +0000  0  NULL  NULL
        	            	857674481994727425  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 09:58:00.100976 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:00.100976 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.100976 +0000 +0000  0  NULL  NULL
        	            	857674482150277121  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-18 09:58:00.148441 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:00.148441 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.148441 +0000 +0000  0  NULL  NULL
        	            	857674482294554625  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 09:58:00.192469 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:00.192469 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.192469 +0000 +0000  0  NULL  NULL
        	            	857674482472845313  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 09:58:00.246885 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:00.246885 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.246885 +0000 +0000  0  NULL  NULL
        	            	857674482627182593  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-18 09:58:00.293981 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:00.293981 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.293981 +0000 +0000  0  NULL  NULL
        	            	857674482774278145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:58:00.338881 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:04.406962 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.338881 +0000 +0000  0  NULL  NULL
        	            	857674482933235713  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:58:00.387393 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:04.479369 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.387393 +0000 +0000  0  NULL  NULL
        	            	857674483078332417  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:58:00.431658 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:04.55435 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.431658 +0000 +0000  0  NULL  NULL
        	            	857674483223199745  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 09:58:00.47587 +0000 +0000   NULL                                    NULL                                    2023-04-18 09:58:00.47587 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.47587 +0000 +0000   0  NULL  NULL
        	            	857674483366821889  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 09:58:00.519694 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:00.519694 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.519694 +0000 +0000  0  NULL  NULL
        	            	857674483509559297  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-18 09:58:00.563262 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:00.563262 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.563262 +0000 +0000  0  NULL  NULL
        	            	857674483674021889  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 09:58:00.613451 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:00.613451 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.613451 +0000 +0000  0  NULL  NULL
        	            	857674483810435073  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 09:58:00.655077 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:00.655077 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.655077 +0000 +0000  0  NULL  NULL
        	            	857674483947208705  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-18 09:58:00.696821 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:00.696821 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.696821 +0000 +0000  0  NULL  NULL
        	            	857674484080246785  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 09:58:00.737432 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:00.737432 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.737432 +0000 +0000  0  NULL  NULL
        	            	857674484223934465  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 09:58:00.781269 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:00.781269 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.781269 +0000 +0000  0  NULL  NULL
        	            	857674484365066241  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-18 09:58:00.824339 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:00.824339 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.824339 +0000 +0000  0  NULL  NULL
        	            	857674484502265857  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:58:00.866224 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:00.866224 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.866224 +0000 +0000  0  NULL  NULL
        	            	857674484649066497  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:58:00.911008 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:00.911008 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.911008 +0000 +0000  0  NULL  NULL
        	            	857674484780695553  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-18 09:58:00.951185 +0000 +0000  NULL                                    NULL                                    2023-04-18 09:58:00.951185 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-18 09:58:00.951185 +0000 +0000  0  NULL  NULL
        	            	857674485322416129  AUTO CREATE STATS                Table statistics refresh for system.public.database_role_settings                       CREATE STATISTICS __auto__ FROM [44] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-18 09:58:01.116472 +0000 +0000  2023-04-18 09:58:01.218969 +0000 +0000  NULL                                    2023-04-18 09:58:01.184285 +0000 +0000  0  NULL    1     6910131217459716333  2023-04-18 09:58:01.218975 +0000 +0000  2023-04-19 09:58:01.218975 +0000 +0000  1  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (10.19s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ dc5c429acb69496f1f7678bf41e608c05f1062d8:

        	            	857968142050000897  BACKUP                           fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-19 10:51:38.037753 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:38.037753 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:38.037753 +0000 +0000  0  NULL  NULL
        	            	857968142282555393  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-19 10:51:38.108716 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:38.108716 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:38.108716 +0000 +0000  0  NULL  NULL
        	            	857968142530150401  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-19 10:51:38.184293 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:38.184293 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:38.184293 +0000 +0000  0  NULL  NULL
        	            	857968142775058433  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-19 10:51:38.259016 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:38.259016 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:38.259016 +0000 +0000  0  NULL  NULL
        	            	857968143011545089  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-19 10:51:38.331191 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:38.331191 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:38.331191 +0000 +0000  0  NULL  NULL
        	            	857968143246557185  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-19 10:51:38.402916 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:38.402916 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:38.402916 +0000 +0000  0  NULL  NULL
        	            	857968143482617857  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-19 10:51:38.474952 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:38.474952 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:38.474952 +0000 +0000  0  NULL  NULL
        	            	857968143709011969  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-19 10:51:38.544032 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:38.544032 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:38.544032 +0000 +0000  0  NULL  NULL
        	            	857968143727001601  AUTO CREATE STATS                Table statistics refresh for system.public.statement_bundle_chunks                      CREATE STATISTICS __auto__ FROM [34] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-19 10:51:38.549512 +0000 +0000  2023-04-19 10:51:38.69847 +0000 +0000   2023-04-19 10:51:39.248361 +0000 +0000  2023-04-19 10:51:39.248417 +0000 +0000  1  NULL    1     9049129211748445085  2023-04-19 10:51:38.698488 +0000 +0000  2023-04-20 10:51:38.698488 +0000 +0000  1  NULL  NULL
        	            	857968143963291649  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-19 10:51:38.621641 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:38.621641 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:38.621641 +0000 +0000  0  NULL  NULL
        	            	857968144196861953  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-19 10:51:38.692917 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:38.692917 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:38.692917 +0000 +0000  0  NULL  NULL
        	            	857968144423059457  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-19 10:51:38.761947 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:38.761947 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:38.761947 +0000 +0000  0  NULL  NULL
        	            	857968144647094273  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-19 10:51:38.830316 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:38.830316 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:38.830316 +0000 +0000  0  NULL  NULL
        	            	857968144882630657  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-19 10:51:38.902201 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:38.902201 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:38.902201 +0000 +0000  0  NULL  NULL
        	            	857968145109123073  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-19 10:51:38.971325 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:38.971325 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:38.971325 +0000 +0000  0  NULL  NULL
        	            	857968145375592449  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-19 10:51:39.052646 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:39.052646 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:39.052646 +0000 +0000  0  NULL  NULL
        	            	857968145636753409  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-19 10:51:39.13233 +0000 +0000   NULL                                    NULL                                    2023-04-19 10:51:39.13233 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:39.13233 +0000 +0000   0  NULL  NULL
        	            	857968145863999489  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 10:51:39.201692 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:39.201692 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:39.201692 +0000 +0000  0  NULL  NULL
        	            	857968146090459137  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 10:51:39.270803 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:39.270803 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:39.270803 +0000 +0000  0  NULL  NULL
        	            	857968146332712961  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 10:51:39.344728 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:39.344728 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:39.344728 +0000 +0000  0  NULL  NULL
        	            	857968146606784513  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-19 10:51:39.428367 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:39.428367 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:39.428367 +0000 +0000  0  NULL  NULL
        	            	857968146838126593  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-19 10:51:39.498965 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:39.498965 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:39.498965 +0000 +0000  0  NULL  NULL
        	            	857968147076251649  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-19 10:51:39.571658 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:39.571658 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:39.571658 +0000 +0000  0  NULL  NULL
        	            	857968147313754113  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-19 10:51:39.644122 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:39.644122 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:39.644122 +0000 +0000  0  NULL  NULL
        	            	857968147585892353  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-19 10:51:39.727163 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:39.727163 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:39.727163 +0000 +0000  0  NULL  NULL
        	            	857968147838861313  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-19 10:51:39.804371 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:39.804371 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:39.804371 +0000 +0000  0  NULL  NULL
        	            	857968147896598529  AUTO CREATE STATS                Table statistics refresh for system.public.tenant_usage                                 CREATE STATISTICS __auto__ FROM [45] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-19 10:51:39.82197 +0000 +0000   2023-04-19 10:51:39.971137 +0000 +0000  2023-04-19 10:51:40.906263 +0000 +0000  2023-04-19 10:51:40.906323 +0000 +0000  1  NULL    1     1701322530031792208  2023-04-19 10:51:39.971153 +0000 +0000  2023-04-20 10:51:39.971153 +0000 +0000  1  NULL  NULL
        	            	857968148074528769  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-19 10:51:39.876295 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:39.876295 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:39.876295 +0000 +0000  0  NULL  NULL
        	            	857968148315701249  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-19 10:51:39.949894 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:39.949894 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:39.949894 +0000 +0000  0  NULL  NULL
        	            	857968148556873729  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-19 10:51:40.023491 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:40.023491 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:40.023491 +0000 +0000  0  NULL  NULL
        	            	857968148790050817  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-19 10:51:40.094655 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:40.094655 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:40.094655 +0000 +0000  0  NULL  NULL
        	            	857968149044625409  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-19 10:51:40.17234 +0000 +0000   NULL                                    NULL                                    2023-04-19 10:51:40.17234 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:40.17234 +0000 +0000   0  NULL  NULL
        	            	857968149293268993  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-19 10:51:40.248236 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:40.248236 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:40.248236 +0000 +0000  0  NULL  NULL
        	            	857968149573337089  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-19 10:51:40.333685 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:40.333685 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:40.333685 +0000 +0000  0  NULL  NULL
        	            	857968149812871169  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-19 10:51:40.406795 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:40.406795 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:40.406795 +0000 +0000  0  NULL  NULL
        	            	857968150048047105  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-19 10:51:40.478563 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:40.478563 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:40.478563 +0000 +0000  0  NULL  NULL
        	            	857968150286073857  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-19 10:51:40.551196 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:40.551196 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:40.551196 +0000 +0000  0  NULL  NULL
        	            	857968150525214721  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-19 10:51:40.624181 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:40.624181 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:40.624181 +0000 +0000  0  NULL  NULL
        	            	857968150762586113  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-19 10:51:40.696617 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:40.696617 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:40.696617 +0000 +0000  0  NULL  NULL
        	            	857968150991142913  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-19 10:51:40.766366 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:40.766366 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:40.766366 +0000 +0000  0  NULL  NULL
        	            	857968151228350465  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-19 10:51:40.838754 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:40.838754 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:40.838754 +0000 +0000  0  NULL  NULL
        	            	857968151464411137  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-19 10:51:40.910799 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:40.910799 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:40.910799 +0000 +0000  0  NULL  NULL
        	            	857968151715676161  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-19 10:51:40.987477 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:40.987477 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:40.987477 +0000 +0000  0  NULL  NULL
        	            	857968151946166273  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-19 10:51:41.057827 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:41.057827 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:41.057827 +0000 +0000  0  NULL  NULL
        	            	857968152179310593  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-19 10:51:41.128973 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:41.128973 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:41.128973 +0000 +0000  0  NULL  NULL
        	            	857968152415862785  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 10:51:41.201172 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:41.201172 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:41.201172 +0000 +0000  0  NULL  NULL
        	            	857968152637374465  AUTO CREATE STATS                Table statistics refresh for system.public.role_id_seq                                  CREATE STATISTICS __auto__ FROM [48] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-19 10:51:41.268748 +0000 +0000  2023-04-19 10:51:41.420761 +0000 +0000  NULL                                    2023-04-19 10:51:41.367469 +0000 +0000  0  NULL    1     5636351735210913871  2023-04-19 10:51:41.420775 +0000 +0000  2023-04-20 10:51:41.420775 +0000 +0000  1  NULL  NULL
        	            	857968152655069185  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 10:51:41.274165 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:41.274165 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:41.274165 +0000 +0000  0  NULL  NULL
        	            	857968152893390849  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-19 10:51:41.346889 +0000 +0000  NULL                                    NULL                                    2023-04-19 10:51:41.346889 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-19 10:51:41.346889 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/resume_ALL_BACKUP_JOBS (8.39s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ 4524616140097c0f6b921e5d1e94ebd405f0809f:

        	            	858248468785659905  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 10:37:26.968313 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:26.968313 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:26.968313 +0000 +0000  0  NULL  NULL
        	            	858248469041217537  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 10:37:27.0463 +0000 +0000    NULL                                    NULL                                    2023-04-20 10:37:27.0463 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:27.0463 +0000 +0000    0  NULL  NULL
        	            	858248469271969793  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 10:37:27.11672 +0000 +0000   NULL                                    NULL                                    2023-04-20 10:37:27.11672 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:27.11672 +0000 +0000   0  NULL  NULL
        	            	858248469504000001  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 10:37:27.187524 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:27.187524 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:27.187524 +0000 +0000  0  NULL  NULL
        	            	858248469729411073  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-20 10:37:27.25631 +0000 +0000   NULL                                    NULL                                    2023-04-20 10:37:27.25631 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:27.25631 +0000 +0000   0  NULL  NULL
        	            	858248469954363393  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-20 10:37:27.324968 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:27.324968 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:27.324968 +0000 +0000  0  NULL  NULL
        	            	858248470183280641  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-20 10:37:27.394833 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:27.394833 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:27.394833 +0000 +0000  0  NULL  NULL
        	            	858248470457090049  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 10:37:27.478403 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:27.478403 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:27.478403 +0000 +0000  0  NULL  NULL
        	            	858248470697771009  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 10:37:27.551835 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:27.551835 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:27.551835 +0000 +0000  0  NULL  NULL
        	            	858248470934028289  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 10:37:27.623933 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:27.623933 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:27.623933 +0000 +0000  0  NULL  NULL
        	            	858248471175168001  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 10:37:27.697537 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:27.697537 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:27.697537 +0000 +0000  0  NULL  NULL
        	            	858248471212851201  AUTO CREATE STATS                Table statistics refresh for system.public.comments                                     CREATE STATISTICS __auto__ FROM [24] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-20 10:37:27.709012 +0000 +0000  2023-04-20 10:37:27.870402 +0000 +0000  2023-04-20 10:37:28.53325 +0000 +0000   2023-04-20 10:37:28.533301 +0000 +0000  1  NULL    1     3193909719138766071  2023-04-20 10:37:27.870419 +0000 +0000  2023-04-21 10:37:27.870419 +0000 +0000  1  NULL  NULL
        	            	858248471413850113  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 10:37:27.770362 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:27.770362 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:27.770362 +0000 +0000  0  NULL  NULL
        	            	858248471710695425  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 10:37:27.860964 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:27.860964 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:27.860964 +0000 +0000  0  NULL  NULL
        	            	858248471947509761  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 10:37:27.933252 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:27.933252 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:27.933252 +0000 +0000  0  NULL  NULL
        	            	858248472186847233  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 10:37:28.006271 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:28.006271 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:28.006271 +0000 +0000  0  NULL  NULL
        	            	858248472403804161  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 10:37:28.072492 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:28.072492 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:28.072492 +0000 +0000  0  NULL  NULL
        	            	858248472676007937  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 10:37:28.155542 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:28.155542 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:28.155542 +0000 +0000  0  NULL  NULL
        	            	858248472912920577  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 10:37:28.227843 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:28.227843 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:28.227843 +0000 +0000  0  NULL  NULL
        	            	858248473153994753  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 10:37:28.301409 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:28.301409 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:28.301409 +0000 +0000  0  NULL  NULL
        	            	858248473419874305  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 10:37:28.382561 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:31.458803 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:28.382561 +0000 +0000  0  NULL  NULL
        	            	858248473643778049  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 10:37:28.450889 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:31.541133 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:28.450889 +0000 +0000  0  NULL  NULL
        	            	858248473888915457  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 10:37:28.525702 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:31.625032 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:28.525702 +0000 +0000  0  NULL  NULL
        	            	858248474129465345  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 10:37:28.599104 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:32.011446 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:28.599104 +0000 +0000  0  NULL  NULL
        	            	858248474353238017  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 10:37:28.667407 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:32.096677 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:28.667407 +0000 +0000  0  NULL  NULL
        	            	858248474638352385  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 10:37:28.754413 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:32.181486 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:28.754413 +0000 +0000  0  NULL  NULL
        	            	858248474860814337  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 10:37:28.822293 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:28.822293 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:28.822293 +0000 +0000  0  NULL  NULL
        	            	858248475093532673  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 10:37:28.893352 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:28.893352 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:28.893352 +0000 +0000  0  NULL  NULL
        	            	858248475117780993  AUTO CREATE STATS                Table statistics refresh for system.public.statement_diagnostics_requests               CREATE STATISTICS __auto__ FROM [35] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-20 10:37:28.900709 +0000 +0000  2023-04-20 10:37:29.05989 +0000 +0000   2023-04-20 10:37:29.828889 +0000 +0000  2023-04-20 10:37:29.828947 +0000 +0000  1  NULL    1     3815918329399630728  2023-04-20 10:37:29.059906 +0000 +0000  2023-04-21 10:37:29.059906 +0000 +0000  1  NULL  NULL
        	            	858248475324448769  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 10:37:28.963797 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:28.963797 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:28.963797 +0000 +0000  0  NULL  NULL
        	            	858248475566178305  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 10:37:29.03756 +0000 +0000   NULL                                    NULL                                    2023-04-20 10:37:29.03756 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:29.03756 +0000 +0000   0  NULL  NULL
        	            	858248475796799489  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 10:37:29.107942 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:29.107942 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:29.107942 +0000 +0000  0  NULL  NULL
        	            	858248476027092993  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 10:37:29.178215 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:29.178215 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:29.178215 +0000 +0000  0  NULL  NULL
        	            	858248476254568449  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 10:37:29.247642 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:31.7577 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:29.247642 +0000 +0000  0  NULL  NULL
        	            	858248476494200833  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 10:37:29.320764 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:31.839139 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:29.320764 +0000 +0000  0  NULL  NULL
        	            	858248476722823169  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 10:37:29.390535 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:31.922388 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:29.390535 +0000 +0000  0  NULL  NULL
        	            	858248476988243969  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 10:37:29.471546 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:29.471546 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:29.471546 +0000 +0000  0  NULL  NULL
        	            	858248477214408705  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 10:37:29.540552 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:29.540552 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:29.540552 +0000 +0000  0  NULL  NULL
        	            	858248477446209537  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 10:37:29.611298 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:29.611298 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:29.611298 +0000 +0000  0  NULL  NULL
        	            	858248477667786753  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 10:37:29.678916 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:29.678916 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:29.678916 +0000 +0000  0  NULL  NULL
        	            	858248477890805761  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 10:37:29.746983 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:29.746983 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:29.746983 +0000 +0000  0  NULL  NULL
        	            	858248478125555713  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 10:37:29.818613 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:29.818613 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:29.818613 +0000 +0000  0  NULL  NULL
        	            	858248478362599425  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 10:37:29.89097 +0000 +0000   NULL                                    NULL                                    2023-04-20 10:37:29.89097 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:29.89097 +0000 +0000   0  NULL  NULL
        	            	858248478600364033  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 10:37:29.963525 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:29.963525 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:29.963525 +0000 +0000  0  NULL  NULL
        	            	858248478848745473  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 10:37:30.039322 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:30.039322 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:30.039322 +0000 +0000  0  NULL  NULL
        	            	858248479089786881  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 10:37:30.112887 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:30.112887 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:30.112887 +0000 +0000  0  NULL  NULL
        	            	858248479342723073  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 10:37:30.190067 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:30.190067 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:30.190067 +0000 +0000  0  NULL  NULL
        	            	858248479495880705  AUTO CREATE STATS                Table statistics refresh for system.public.statement_diagnostics                        CREATE STATISTICS __auto__ FROM [36] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-20 10:37:30.236794 +0000 +0000  2023-04-20 10:37:30.398326 +0000 +0000  NULL                                    2023-04-20 10:37:30.336074 +0000 +0000  0  NULL    1     749361081573973244   2023-04-20 10:37:30.398342 +0000 +0000  2023-04-21 10:37:30.398342 +0000 +0000  1  NULL  NULL
        	            	858248479570558977  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 10:37:30.259599 +0000 +0000  NULL                                    NULL                                    2023-04-20 10:37:30.259599 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 10:37:30.259599 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (10.24s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ 4524616140097c0f6b921e5d1e94ebd405f0809f:

        	            	858257452735660033  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 11:23:08.652274 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:08.652274 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:08.652274 +0000 +0000  0  NULL  NULL
        	            	858257452883247105  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 11:23:08.69733 +0000 +0000   NULL                                    NULL                                    2023-04-20 11:23:08.69733 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:08.69733 +0000 +0000   0  NULL  NULL
        	            	858257453045415937  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 11:23:08.746831 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:08.746831 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:08.746831 +0000 +0000  0  NULL  NULL
        	            	858257453211516929  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 11:23:08.797499 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:08.797499 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:08.797499 +0000 +0000  0  NULL  NULL
        	            	858257453361790977  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 11:23:08.84336 +0000 +0000   NULL                                    NULL                                    2023-04-20 11:23:08.84336 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:08.84336 +0000 +0000   0  NULL  NULL
        	            	858257453525794817  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 11:23:08.893404 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:08.893404 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:08.893404 +0000 +0000  0  NULL  NULL
        	            	858257453667188737  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-20 11:23:08.936554 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:08.936554 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:08.936554 +0000 +0000  0  NULL  NULL
        	            	858257453802946561  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-20 11:23:08.97799 +0000 +0000   NULL                                    NULL                                    2023-04-20 11:23:08.97799 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:08.97799 +0000 +0000   0  NULL  NULL
        	            	858257453941129217  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-20 11:23:09.020154 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:09.020154 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.020154 +0000 +0000  0  NULL  NULL
        	            	858257454085013505  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 11:23:09.064065 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:09.064065 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.064065 +0000 +0000  0  NULL  NULL
        	            	858257454237548545  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 11:23:09.110614 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:09.110614 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.110614 +0000 +0000  0  NULL  NULL
        	            	858257454398930945  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 11:23:09.159866 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:09.159866 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.159866 +0000 +0000  0  NULL  NULL
        	            	858257454540652545  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 11:23:09.203114 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:09.203114 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.203114 +0000 +0000  0  NULL  NULL
        	            	858257454681522177  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 11:23:09.246115 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:09.246115 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.246115 +0000 +0000  0  NULL  NULL
        	            	858257454817968129  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 11:23:09.287746 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:09.287746 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.287746 +0000 +0000  0  NULL  NULL
        	            	858257454952808449  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 11:23:09.328889 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:09.328889 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.328889 +0000 +0000  0  NULL  NULL
        	            	858257455087288321  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 11:23:09.369936 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:09.369936 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.369936 +0000 +0000  0  NULL  NULL
        	            	858257455225241601  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 11:23:09.412036 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:09.412036 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.412036 +0000 +0000  0  NULL  NULL
        	            	858257455363620865  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 11:23:09.45427 +0000 +0000   NULL                                    NULL                                    2023-04-20 11:23:09.45427 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.45427 +0000 +0000   0  NULL  NULL
        	            	858257455510880257  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 11:23:09.499208 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:09.499208 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.499208 +0000 +0000  0  NULL  NULL
        	            	858257455676030977  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 11:23:09.549607 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:09.549607 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.549607 +0000 +0000  0  NULL  NULL
        	            	858257455815229441  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 11:23:09.592093 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:13.294036 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.592093 +0000 +0000  0  NULL  NULL
        	            	858257455953707009  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 11:23:09.634352 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:13.365053 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.634352 +0000 +0000  0  NULL  NULL
        	            	858257456094773249  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 11:23:09.677401 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:13.436004 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.677401 +0000 +0000  0  NULL  NULL
        	            	858257456230465537  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 11:23:09.718818 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:14.176464 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.718818 +0000 +0000  0  NULL  NULL
        	            	858257456389718017  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 11:23:09.767409 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:14.245122 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.767409 +0000 +0000  0  NULL  NULL
        	            	858257456561553409  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 11:23:09.819862 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:14.314464 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.819862 +0000 +0000  0  NULL  NULL
        	            	858257456751837185  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 11:23:09.877915 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:09.877915 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.877915 +0000 +0000  0  NULL  NULL
        	            	858257456892411905  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 11:23:09.920833 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:09.920833 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.920833 +0000 +0000  0  NULL  NULL
        	            	858257457032953857  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-20 11:23:09.963721 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:09.963721 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:09.963721 +0000 +0000  0  NULL  NULL
        	            	858257457169203201  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 11:23:10.005295 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:10.005295 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:10.005295 +0000 +0000  0  NULL  NULL
        	            	858257457301979137  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 11:23:10.045806 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:10.045806 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:10.045806 +0000 +0000  0  NULL  NULL
        	            	858257457440587777  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-20 11:23:10.08812 +0000 +0000   NULL                                    NULL                                    2023-04-20 11:23:10.08812 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:10.08812 +0000 +0000   0  NULL  NULL
        	            	858257457582604289  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 11:23:10.131443 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:13.948722 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:10.131443 +0000 +0000  0  NULL  NULL
        	            	858257457722261505  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 11:23:10.17407 +0000 +0000   NULL                                    NULL                                    2023-04-20 11:23:14.021325 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:10.17407 +0000 +0000   0  NULL  NULL
        	            	858257457859264513  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 11:23:10.215892 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:14.092271 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:10.215892 +0000 +0000  0  NULL  NULL
        	            	858257457996300289  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 11:23:10.257698 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:10.257698 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:10.257698 +0000 +0000  0  NULL  NULL
        	            	858257458136252417  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 11:23:10.300409 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:10.300409 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:10.300409 +0000 +0000  0  NULL  NULL
        	            	858257458280595457  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-20 11:23:10.344462 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:10.344462 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:10.344462 +0000 +0000  0  NULL  NULL
        	            	858257458422317057  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 11:23:10.387717 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:10.387717 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:10.387717 +0000 +0000  0  NULL  NULL
        	            	858257458567774209  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 11:23:10.432093 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:10.432093 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:10.432093 +0000 +0000  0  NULL  NULL
        	            	858257458697404417  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-20 11:23:10.471658 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:10.471658 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:10.471658 +0000 +0000  0  NULL  NULL
        	            	858257458833883137  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 11:23:10.513309 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:10.513309 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:10.513309 +0000 +0000  0  NULL  NULL
        	            	858257458978127873  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 11:23:10.557336 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:10.557336 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:10.557336 +0000 +0000  0  NULL  NULL
        	            	858257459118899201  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-20 11:23:10.600306 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:10.600306 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:10.600306 +0000 +0000  0  NULL  NULL
        	            	858257459255738369  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 11:23:10.642049 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:10.642049 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:10.642049 +0000 +0000  0  NULL  NULL
        	            	858257459387858945  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 11:23:10.682375 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:10.682375 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:10.682375 +0000 +0000  0  NULL  NULL
        	            	858257459529318401  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-20 11:23:10.725549 +0000 +0000  NULL                                    NULL                                    2023-04-20 11:23:10.725549 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-20 11:23:10.725549 +0000 +0000  0  NULL  NULL
        	            	858257459895074817  AUTO CREATE STATS                Table statistics refresh for system.public.role_members                                 CREATE STATISTICS __auto__ FROM [23] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-20 11:23:10.837131 +0000 +0000  2023-04-20 11:23:10.952808 +0000 +0000  NULL                                    2023-04-20 11:23:10.917936 +0000 +0000  0  NULL    1     4712226755547353316  2023-04-20 11:23:10.952813 +0000 +0000  2023-04-21 11:23:10.952813 +0000 +0000  1  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (9.84s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ 909577e7be65a7b38fa61aa2e2afb536b4a595eb:

        	            	858529399080091649  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-21 10:26:20.090388 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:20.090388 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:20.090388 +0000 +0000  0  NULL  NULL
        	            	858529399311597569  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 10:26:20.161033 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:20.161033 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:20.161033 +0000 +0000  0  NULL  NULL
        	            	858529399538974721  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 10:26:20.230427 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:20.230427 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:20.230427 +0000 +0000  0  NULL  NULL
        	            	858529399782178817  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 10:26:20.304649 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:20.304649 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:20.304649 +0000 +0000  0  NULL  NULL
        	            	858529400057954305  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-21 10:26:20.388811 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:20.388811 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:20.388811 +0000 +0000  0  NULL  NULL
        	            	858529400289918977  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-21 10:26:20.4596 +0000 +0000    NULL                                    NULL                                    2023-04-21 10:26:20.4596 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:20.4596 +0000 +0000    0  NULL  NULL
        	            	858529400520015873  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-21 10:26:20.529816 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:20.529816 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:20.529816 +0000 +0000  0  NULL  NULL
        	            	858529400760401921  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 10:26:20.603177 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:20.603177 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:20.603177 +0000 +0000  0  NULL  NULL
        	            	858529400995971073  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 10:26:20.675073 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:20.675073 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:20.675073 +0000 +0000  0  NULL  NULL
        	            	858529401245892609  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 10:26:20.751354 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:20.751354 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:20.751354 +0000 +0000  0  NULL  NULL
        	            	858529401499418625  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 10:26:20.828704 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:20.828704 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:20.828704 +0000 +0000  0  NULL  NULL
        	            	858529401731219457  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 10:26:20.899446 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:20.899446 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:20.899446 +0000 +0000  0  NULL  NULL
        	            	858529401966329857  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 10:26:20.971196 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:20.971196 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:20.971196 +0000 +0000  0  NULL  NULL
        	            	858529402144817153  AUTO CREATE STATS                Table statistics refresh for system.public.eventlog                                     CREATE STATISTICS __auto__ FROM [12] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-21 10:26:21.025657 +0000 +0000  2023-04-21 10:26:21.171346 +0000 +0000  2023-04-21 10:26:21.886852 +0000 +0000  2023-04-21 10:26:21.886911 +0000 +0000  1  NULL    1     8850925875633092016  2023-04-21 10:26:21.171361 +0000 +0000  2023-04-22 10:26:21.171361 +0000 +0000  1  NULL  NULL
        	            	858529402187481089  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 10:26:21.038681 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:21.038681 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:21.038681 +0000 +0000  0  NULL  NULL
        	            	858529402419937281  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 10:26:21.109635 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:21.109635 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:21.109635 +0000 +0000  0  NULL  NULL
        	            	858529402647609345  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 10:26:21.179105 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:21.179105 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:21.179105 +0000 +0000  0  NULL  NULL
        	            	858529402866794497  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:26:21.246002 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:21.246002 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:21.246002 +0000 +0000  0  NULL  NULL
        	            	858529403097841665  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:26:21.316514 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:21.316514 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:21.316514 +0000 +0000  0  NULL  NULL
        	            	858529403354218497  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:26:21.39475 +0000 +0000   NULL                                    NULL                                    2023-04-21 10:26:21.39475 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:21.39475 +0000 +0000   0  NULL  NULL
        	            	858529403625996289  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:26:21.477689 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:24.557913 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:21.477689 +0000 +0000  0  NULL  NULL
        	            	858529403859173377  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:26:21.54885 +0000 +0000   NULL                                    NULL                                    2023-04-21 10:26:24.651243 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:21.54885 +0000 +0000   0  NULL  NULL
        	            	858529404115615745  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:26:21.627106 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:24.735761 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:21.627106 +0000 +0000  0  NULL  NULL
        	            	858529404396699649  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:26:21.712883 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:25.138548 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:21.712883 +0000 +0000  0  NULL  NULL
        	            	858529404617162753  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:26:21.780166 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:25.223872 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:21.780166 +0000 +0000  0  NULL  NULL
        	            	858529404842672129  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:26:21.849003 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:25.305022 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:21.849003 +0000 +0000  0  NULL  NULL
        	            	858529405075423233  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-21 10:26:21.920013 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:21.920013 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:21.920013 +0000 +0000  0  NULL  NULL
        	            	858529405309321217  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-21 10:26:21.991396 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:21.991396 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:21.991396 +0000 +0000  0  NULL  NULL
        	            	858529405533454337  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-21 10:26:22.059795 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:22.059795 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:22.059795 +0000 +0000  0  NULL  NULL
        	            	858529405776134145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 10:26:22.133866 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:22.133866 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:22.133866 +0000 +0000  0  NULL  NULL
        	            	858529406003085313  AUTO CREATE STATS                Table statistics refresh for system.public.settings                                     CREATE STATISTICS __auto__ FROM [6] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'   root      {}  succeeded         NULL  2023-04-21 10:26:22.203103 +0000 +0000  2023-04-21 10:26:22.349853 +0000 +0000  2023-04-21 10:26:22.9513 +0000 +0000    2023-04-21 10:26:22.951337 +0000 +0000  1  NULL    1     2725216766608838027  2023-04-21 10:26:22.349869 +0000 +0000  2023-04-22 10:26:22.349869 +0000 +0000  1  NULL  NULL
        	            	858529406010425345  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 10:26:22.205355 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:22.205355 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:22.205355 +0000 +0000  0  NULL  NULL
        	            	858529406236327937  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 10:26:22.2743 +0000 +0000    NULL                                    NULL                                    2023-04-21 10:26:22.2743 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:22.2743 +0000 +0000    0  NULL  NULL
        	            	858529406470258689  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:26:22.34569 +0000 +0000   NULL                                    NULL                                    2023-04-21 10:26:24.867294 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:22.34569 +0000 +0000   0  NULL  NULL
        	            	858529406700322817  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:26:22.415895 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:24.946564 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:22.415895 +0000 +0000  0  NULL  NULL
        	            	858529406931271681  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:26:22.486384 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:25.045897 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:22.486384 +0000 +0000  0  NULL  NULL
        	            	858529407166447617  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 10:26:22.558144 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:22.558144 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:22.558144 +0000 +0000  0  NULL  NULL
        	            	858529407398707201  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 10:26:22.629023 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:22.629023 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:22.629023 +0000 +0000  0  NULL  NULL
        	            	858529407629524993  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 10:26:22.699472 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:22.699472 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:22.699472 +0000 +0000  0  NULL  NULL
        	            	858529407901925377  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 10:26:22.7826 +0000 +0000    NULL                                    NULL                                    2023-04-21 10:26:22.7826 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:22.7826 +0000 +0000    0  NULL  NULL
        	            	858529408148045825  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 10:26:22.85771 +0000 +0000   NULL                                    NULL                                    2023-04-21 10:26:22.85771 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:22.85771 +0000 +0000   0  NULL  NULL
        	            	858529408386760705  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 10:26:22.930563 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:22.930563 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:22.930563 +0000 +0000  0  NULL  NULL
        	            	858529408615350273  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 10:26:23.000316 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:23.000316 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:23.000316 +0000 +0000  0  NULL  NULL
        	            	858529408852492289  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 10:26:23.072695 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:23.072695 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:23.072695 +0000 +0000  0  NULL  NULL
        	            	858529409089601537  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 10:26:23.145051 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:23.145051 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:23.145051 +0000 +0000  0  NULL  NULL
        	            	858529409318912001  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:26:23.215028 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:23.215028 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:23.215028 +0000 +0000  0  NULL  NULL
        	            	858529409540227073  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:26:23.282583 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:23.282583 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:23.282583 +0000 +0000  0  NULL  NULL
        	            	858529409666449409  AUTO CREATE STATS                Table statistics refresh for system.public.rangelog                                     CREATE STATISTICS __auto__ FROM [13] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-21 10:26:23.321073 +0000 +0000  2023-04-21 10:26:23.463699 +0000 +0000  NULL                                    2023-04-21 10:26:23.417902 +0000 +0000  0  NULL    1     1892974899553045855  2023-04-21 10:26:23.463716 +0000 +0000  2023-04-22 10:26:23.463716 +0000 +0000  1  NULL  NULL
        	            	858529409780416513  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:26:23.355869 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:26:23.355869 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:26:23.355869 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (10.22s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ 909577e7be65a7b38fa61aa2e2afb536b4a595eb:

        	            	858532456163311617  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-21 10:41:53.038165 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:53.038165 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.038165 +0000 +0000  0  NULL  NULL
        	            	858532456315944961  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-21 10:41:53.084734 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:53.084734 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.084734 +0000 +0000  0  NULL  NULL
        	            	858532456460091393  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-21 10:41:53.128737 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:53.128737 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.128737 +0000 +0000  0  NULL  NULL
        	            	858532456514453505  AUTO CREATE STATS                Table statistics refresh for system.public.replication_stats                            CREATE STATISTICS __auto__ FROM [27] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-21 10:41:53.145293 +0000 +0000  2023-04-21 10:41:53.250863 +0000 +0000  2023-04-21 10:41:55.536476 +0000 +0000  2023-04-21 10:41:55.536617 +0000 +0000  1  NULL    1     2060805991001852873  2023-04-21 10:41:53.250868 +0000 +0000  2023-04-22 10:41:53.250868 +0000 +0000  1  NULL  NULL
        	            	858532456607350785  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 10:41:53.173661 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:53.173661 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.173661 +0000 +0000  0  NULL  NULL
        	            	858532456751529985  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 10:41:53.217711 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:53.217711 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.217711 +0000 +0000  0  NULL  NULL
        	            	858532456895774721  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 10:41:53.261684 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:53.261684 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.261684 +0000 +0000  0  NULL  NULL
        	            	858532457041199105  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-21 10:41:53.306077 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:53.306077 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.306077 +0000 +0000  0  NULL  NULL
        	            	858532457254354945  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-21 10:41:53.371126 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:53.371126 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.371126 +0000 +0000  0  NULL  NULL
        	            	858532457398632449  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-21 10:41:53.415147 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:53.415147 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.415147 +0000 +0000  0  NULL  NULL
        	            	858532457594716161  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 10:41:53.474989 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:53.474989 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.474989 +0000 +0000  0  NULL  NULL
        	            	858532457866133505  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 10:41:53.557819 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:53.557819 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.557819 +0000 +0000  0  NULL  NULL
        	            	858532458110091265  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 10:41:53.63227 +0000 +0000   NULL                                    NULL                                    2023-04-21 10:41:53.63227 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.63227 +0000 +0000   0  NULL  NULL
        	            	858532458328457217  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 10:41:53.698903 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:53.698903 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.698903 +0000 +0000  0  NULL  NULL
        	            	858532458506027009  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 10:41:53.753107 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:53.753107 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.753107 +0000 +0000  0  NULL  NULL
        	            	858532458650861569  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 10:41:53.797295 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:53.797295 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.797295 +0000 +0000  0  NULL  NULL
        	            	858532458783735809  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 10:41:53.837859 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:53.837859 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.837859 +0000 +0000  0  NULL  NULL
        	            	858532458920968193  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 10:41:53.87973 +0000 +0000   NULL                                    NULL                                    2023-04-21 10:41:53.87973 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.87973 +0000 +0000   0  NULL  NULL
        	            	858532459063246849  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 10:41:53.923163 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:53.923163 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.923163 +0000 +0000  0  NULL  NULL
        	            	858532459201560577  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:41:53.965364 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:53.965364 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:53.965364 +0000 +0000  0  NULL  NULL
        	            	858532459357732865  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:41:54.013018 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.013018 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.013018 +0000 +0000  0  NULL  NULL
        	            	858532459497750529  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:41:54.055743 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.055743 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.055743 +0000 +0000  0  NULL  NULL
        	            	858532459640553473  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-21 10:41:54.099334 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.099334 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.099334 +0000 +0000  0  NULL  NULL
        	            	858532459789811713  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-21 10:41:54.144897 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.144897 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.144897 +0000 +0000  0  NULL  NULL
        	            	858532459926618113  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-21 10:41:54.186648 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.186648 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.186648 +0000 +0000  0  NULL  NULL
        	            	858532460069486593  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-21 10:41:54.230231 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.230231 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.230231 +0000 +0000  0  NULL  NULL
        	            	858532460203999233  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-21 10:41:54.271275 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.271275 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.271275 +0000 +0000  0  NULL  NULL
        	            	858532460369641473  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-21 10:41:54.321824 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.321824 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.321824 +0000 +0000  0  NULL  NULL
        	            	858532460501991425  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-21 10:41:54.362236 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.362236 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.362236 +0000 +0000  0  NULL  NULL
        	            	858532460641419265  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-21 10:41:54.404772 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.404772 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.404772 +0000 +0000  0  NULL  NULL
        	            	858532460784582657  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-21 10:41:54.448468 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.448468 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.448468 +0000 +0000  0  NULL  NULL
        	            	858532460927975425  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 10:41:54.492221 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.492221 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.492221 +0000 +0000  0  NULL  NULL
        	            	858532461066747905  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 10:41:54.534582 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.534582 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.534582 +0000 +0000  0  NULL  NULL
        	            	858532461212336129  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-21 10:41:54.579002 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.579002 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.579002 +0000 +0000  0  NULL  NULL
        	            	858532461357334529  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-21 10:41:54.623254 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.623254 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.623254 +0000 +0000  0  NULL  NULL
        	            	858532461511376897  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-21 10:41:54.670271 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.670271 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.670271 +0000 +0000  0  NULL  NULL
        	            	858532461650771969  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-21 10:41:54.712811 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.712811 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.712811 +0000 +0000  0  NULL  NULL
        	            	858532461810647041  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 10:41:54.761602 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.761602 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.761602 +0000 +0000  0  NULL  NULL
        	            	858532461943848961  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 10:41:54.802243 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.802243 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.802243 +0000 +0000  0  NULL  NULL
        	            	858532462089633793  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-21 10:41:54.846738 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.846738 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.846738 +0000 +0000  0  NULL  NULL
        	            	858532462247378945  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 10:41:54.894871 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.894871 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.894871 +0000 +0000  0  NULL  NULL
        	            	858532462388346881  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 10:41:54.937899 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.937899 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.937899 +0000 +0000  0  NULL  NULL
        	            	858532462533312513  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-21 10:41:54.982135 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:54.982135 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:54.982135 +0000 +0000  0  NULL  NULL
        	            	858532462670479361  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 10:41:55.023997 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:55.023997 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:55.023997 +0000 +0000  0  NULL  NULL
        	            	858532462813872129  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 10:41:55.067752 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:55.067752 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:55.067752 +0000 +0000  0  NULL  NULL
        	            	858532462959132673  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-21 10:41:55.112079 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:55.112079 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:55.112079 +0000 +0000  0  NULL  NULL
        	            	858532463103344641  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:41:55.156097 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:55.156097 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:55.156097 +0000 +0000  0  NULL  NULL
        	            	858532463253454849  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:41:55.201896 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:55.201896 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:55.201896 +0000 +0000  0  NULL  NULL
        	            	858532463397928961  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-21 10:41:55.245985 +0000 +0000  NULL                                    NULL                                    2023-04-21 10:41:55.245985 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-21 10:41:55.245985 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/resume_ALL_RESTORE_JOBS (8.47s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ ca2fae8b11a761af8ac7e5334aa5642c15dfb42f:

        	            	858812205592444929  BACKUP                           fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-22 10:24:45.788736 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:45.788736 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:45.788736 +0000 +0000  0  NULL  NULL
        	            	858812205784301569  BACKUP                           fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-22 10:24:45.847293 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:45.847293 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:45.847293 +0000 +0000  0  NULL  NULL
        	            	858812205965803521  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 10:24:45.902679 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:45.902679 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:45.902679 +0000 +0000  0  NULL  NULL
        	            	858812206149107713  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 10:24:45.958623 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:45.958623 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:45.958623 +0000 +0000  0  NULL  NULL
        	            	858812206327824385  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 10:24:46.013156 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:46.013156 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:46.013156 +0000 +0000  0  NULL  NULL
        	            	858812206511161345  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-22 10:24:46.069112 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:46.069112 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:46.069112 +0000 +0000  0  NULL  NULL
        	            	858812206696071169  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-22 10:24:46.125544 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:46.125544 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:46.125544 +0000 +0000  0  NULL  NULL
        	            	858812206887567361  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-22 10:24:46.183979 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:46.183979 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:46.183979 +0000 +0000  0  NULL  NULL
        	            	858812207076900865  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 10:24:46.241765 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:46.241765 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:46.241765 +0000 +0000  0  NULL  NULL
        	            	858812207255322625  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 10:24:46.29621 +0000 +0000   NULL                                    NULL                                    2023-04-22 10:24:46.29621 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:46.29621 +0000 +0000   0  NULL  NULL
        	            	858812207435579393  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 10:24:46.351222 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:46.351222 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:46.351222 +0000 +0000  0  NULL  NULL
        	            	858812207621767169  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 10:24:46.408041 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:46.408041 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:46.408041 +0000 +0000  0  NULL  NULL
        	            	858812207801597953  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 10:24:46.462917 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:46.462917 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:46.462917 +0000 +0000  0  NULL  NULL
        	            	858812208004202497  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 10:24:46.524752 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:46.524752 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:46.524752 +0000 +0000  0  NULL  NULL
        	            	858812208178987009  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 10:24:46.578093 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:46.578093 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:46.578093 +0000 +0000  0  NULL  NULL
        	            	858812208366682113  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 10:24:46.635373 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:46.635373 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:46.635373 +0000 +0000  0  NULL  NULL
        	            	858812208538812417  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 10:24:46.687903 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:46.687903 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:46.687903 +0000 +0000  0  NULL  NULL
        	            	858812208723034113  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:24:46.744119 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:46.744119 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:46.744119 +0000 +0000  0  NULL  NULL
        	            	858812208915939329  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:24:46.802993 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:46.802993 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:46.802993 +0000 +0000  0  NULL  NULL
        	            	858812209179688961  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:24:46.883479 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:46.883479 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:46.883479 +0000 +0000  0  NULL  NULL
        	            	858812209435869185  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-22 10:24:46.961657 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:46.961657 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:46.961657 +0000 +0000  0  NULL  NULL
        	            	858812209686675457  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-22 10:24:47.038198 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:47.038198 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:47.038198 +0000 +0000  0  NULL  NULL
        	            	858812209920737281  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pending           NULL  2023-04-22 10:24:47.109632 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:47.109632 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:47.109632 +0000 +0000  0  NULL  NULL
        	            	858812210148835329  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-22 10:24:47.179236 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:47.179236 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:47.179236 +0000 +0000  0  NULL  NULL
        	            	858812210222137345  AUTO CREATE STATS                Table statistics refresh for system.public.reports_meta                                 CREATE STATISTICS __auto__ FROM [28] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-22 10:24:47.201591 +0000 +0000  2023-04-22 10:24:47.372437 +0000 +0000  2023-04-22 10:24:48.165421 +0000 +0000  2023-04-22 10:24:48.165466 +0000 +0000  1  NULL    1     8560121436653243047  2023-04-22 10:24:47.372449 +0000 +0000  2023-04-23 10:24:47.372449 +0000 +0000  1  NULL  NULL
        	            	858812210403442689  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-22 10:24:47.25693 +0000 +0000   NULL                                    NULL                                    2023-04-22 10:24:47.25693 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:47.25693 +0000 +0000   0  NULL  NULL
        	            	858812210686263297  RESTORE                          fake job                                                                                                                                                                           testuser  {}  running           NULL  2023-04-22 10:24:47.34324 +0000 +0000   NULL                                    NULL                                    2023-04-22 10:24:47.34324 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:47.34324 +0000 +0000   0  NULL  NULL
        	            	858812210917048321  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-22 10:24:47.41369 +0000 +0000   NULL                                    NULL                                    2023-04-22 10:24:47.41369 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:47.41369 +0000 +0000   0  NULL  NULL
        	            	858812211138035713  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-22 10:24:47.481115 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:47.481115 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:47.481115 +0000 +0000  0  NULL  NULL
        	            	858812211367116801  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-22 10:24:47.551028 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:47.551028 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:47.551028 +0000 +0000  0  NULL  NULL
        	            	858812211589382145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 10:24:47.618856 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:47.618856 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:47.618856 +0000 +0000  0  NULL  NULL
        	            	858812211809288193  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 10:24:47.685967 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:47.685967 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:47.685967 +0000 +0000  0  NULL  NULL
        	            	858812212047216641  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 10:24:47.758573 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:47.758573 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:47.758573 +0000 +0000  0  NULL  NULL
        	            	858812212283441153  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-22 10:24:47.830665 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:47.830665 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:47.830665 +0000 +0000  0  NULL  NULL
        	            	858812212525400065  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-22 10:24:47.904506 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:47.904506 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:47.904506 +0000 +0000  0  NULL  NULL
        	            	858812212757430273  RESTORE                          fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-22 10:24:47.975319 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:47.975319 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:47.975319 +0000 +0000  0  NULL  NULL
        	            	858812212990836737  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 10:24:48.046542 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:48.046542 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:48.046542 +0000 +0000  0  NULL  NULL
        	            	858812213215199233  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 10:24:48.115019 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:48.115019 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:48.115019 +0000 +0000  0  NULL  NULL
        	            	858812213458141185  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 10:24:48.189161 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:48.189161 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:48.189161 +0000 +0000  0  NULL  NULL
        	            	858812213696397313  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 10:24:48.261938 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:48.261938 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:48.261938 +0000 +0000  0  NULL  NULL
        	            	858812213930393601  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 10:24:48.333281 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:48.333281 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:48.333281 +0000 +0000  0  NULL  NULL
        	            	858812214157115393  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 10:24:48.402471 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:48.402471 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:48.402471 +0000 +0000  0  NULL  NULL
        	            	858812214421979137  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 10:24:48.483368 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:48.483368 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:48.483368 +0000 +0000  0  NULL  NULL
        	            	858812214651256833  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 10:24:48.553264 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:48.553264 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:48.553264 +0000 +0000  0  NULL  NULL
        	            	858812214877356033  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 10:24:48.622271 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:48.622271 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:48.622271 +0000 +0000  0  NULL  NULL
        	            	858812215112302593  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:24:48.693982 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:48.693982 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:48.693982 +0000 +0000  0  NULL  NULL
        	            	858812215349837825  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:24:48.766466 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:48.766466 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:48.766466 +0000 +0000  0  NULL  NULL
        	            	858812215470260225  AUTO CREATE STATS                Table statistics refresh for system.public.join_tokens                                  CREATE STATISTICS __auto__ FROM [41] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-22 10:24:48.803197 +0000 +0000  2023-04-22 10:24:48.974218 +0000 +0000  NULL                                    2023-04-22 10:24:48.92082 +0000 +0000   0  NULL    1     2405304866546397895  2023-04-22 10:24:48.974227 +0000 +0000  2023-04-23 10:24:48.974227 +0000 +0000  1  NULL  NULL
        	            	858812215572725761  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:24:48.834479 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:24:48.834479 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:24:48.834479 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/resume_ALL_BACKUP_JOBS (7.57s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ ca2fae8b11a761af8ac7e5334aa5642c15dfb42f:

        	            	858815642508951553  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-22 10:42:14.652418 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:14.652418 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:14.652418 +0000 +0000  0  NULL  NULL
        	            	858815642635042817  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-22 10:42:14.690906 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:14.690906 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:14.690906 +0000 +0000  0  NULL  NULL
        	            	858815642763132929  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-22 10:42:14.729996 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:14.729996 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:14.729996 +0000 +0000  0  NULL  NULL
        	            	858815642891321345  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 10:42:14.769109 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:14.769109 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:14.769109 +0000 +0000  0  NULL  NULL
        	            	858815643018854401  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 10:42:14.808055 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:14.808055 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:14.808055 +0000 +0000  0  NULL  NULL
        	            	858815643145109505  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 10:42:14.846565 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:14.846565 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:14.846565 +0000 +0000  0  NULL  NULL
        	            	858815643272708097  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-22 10:42:14.885503 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:14.885503 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:14.885503 +0000 +0000  0  NULL  NULL
        	            	858815643399159809  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-22 10:42:14.924096 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:14.924096 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:14.924096 +0000 +0000  0  NULL  NULL
        	            	858815643556118529  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-22 10:42:14.971993 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:14.971993 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:14.971993 +0000 +0000  0  NULL  NULL
        	            	858815643692466177  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 10:42:15.013622 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:15.013622 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.013622 +0000 +0000  0  NULL  NULL
        	            	858815643824848897  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 10:42:15.054008 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:15.054008 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.054008 +0000 +0000  0  NULL  NULL
        	            	858815643961032705  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 10:42:15.095559 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:15.095559 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.095559 +0000 +0000  0  NULL  NULL
        	            	858815644098363393  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 10:42:15.137473 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:15.137473 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.137473 +0000 +0000  0  NULL  NULL
        	            	858815644227567617  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 10:42:15.176918 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:15.176918 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.176918 +0000 +0000  0  NULL  NULL
        	            	858815644369649665  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 10:42:15.220262 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:15.220262 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.220262 +0000 +0000  0  NULL  NULL
        	            	858815644497248257  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 10:42:15.259205 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:15.259205 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.259205 +0000 +0000  0  NULL  NULL
        	            	858815644631236609  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 10:42:15.300099 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:15.300099 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.300099 +0000 +0000  0  NULL  NULL
        	            	858815644764766209  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 10:42:15.340867 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:15.340867 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.340867 +0000 +0000  0  NULL  NULL
        	            	858815644905766913  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:42:15.383874 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:15.383874 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.383874 +0000 +0000  0  NULL  NULL
        	            	858815645043195905  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:42:15.425812 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:15.425812 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.425812 +0000 +0000  0  NULL  NULL
        	            	858815645238034433  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:42:15.485277 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:15.485277 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.485277 +0000 +0000  0  NULL  NULL
        	            	858815645425958913  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:42:15.542635 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:19.383539 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.542635 +0000 +0000  0  NULL  NULL
        	            	858815645564665857  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:42:15.584963 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:19.45683 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.584963 +0000 +0000  0  NULL  NULL
        	            	858815645811212289  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:42:15.660192 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:19.53116 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.660192 +0000 +0000  0  NULL  NULL
        	            	858815645948706817  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:42:15.702164 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:20.295356 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.702164 +0000 +0000  0  NULL  NULL
        	            	858815646078828545  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:42:15.741869 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:20.36753 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.741869 +0000 +0000  0  NULL  NULL
        	            	858815646202560513  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:42:15.779624 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:20.441564 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.779624 +0000 +0000  0  NULL  NULL
        	            	858815646342709249  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-22 10:42:15.822406 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:15.822406 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.822406 +0000 +0000  0  NULL  NULL
        	            	858815646505009153  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-22 10:42:15.871935 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:15.871935 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.871935 +0000 +0000  0  NULL  NULL
        	            	858815646642864129  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-22 10:42:15.913997 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:15.913997 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.913997 +0000 +0000  0  NULL  NULL
        	            	858815646800805889  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 10:42:15.962191 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:15.962191 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:15.962191 +0000 +0000  0  NULL  NULL
        	            	858815646972084225  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 10:42:16.01446 +0000 +0000   NULL                                    NULL                                    2023-04-22 10:42:16.01446 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:16.01446 +0000 +0000   0  NULL  NULL
        	            	858815647118032897  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-22 10:42:16.059021 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:16.059021 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:16.059021 +0000 +0000  0  NULL  NULL
        	            	858815647245762561  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:42:16.097991 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:20.063087 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:16.097991 +0000 +0000  0  NULL  NULL
        	            	858815647373426689  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:42:16.136954 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:20.13602 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:16.136954 +0000 +0000  0  NULL  NULL
        	            	858815647505022977  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:42:16.177109 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:20.208735 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:16.177109 +0000 +0000  0  NULL  NULL
        	            	858815647636422657  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 10:42:16.217216 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:16.217216 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:16.217216 +0000 +0000  0  NULL  NULL
        	            	858815647769427969  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 10:42:16.257804 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:16.257804 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:16.257804 +0000 +0000  0  NULL  NULL
        	            	858815647896862721  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-22 10:42:16.296683 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:16.296683 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:16.296683 +0000 +0000  0  NULL  NULL
        	            	858815648032292865  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 10:42:16.338021 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:16.338021 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:16.338021 +0000 +0000  0  NULL  NULL
        	            	858815648165068801  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 10:42:16.378535 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:16.378535 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:16.378535 +0000 +0000  0  NULL  NULL
        	            	858815648298336257  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-22 10:42:16.419219 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:16.419219 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:16.419219 +0000 +0000  0  NULL  NULL
        	            	858815648430030849  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 10:42:16.4594 +0000 +0000    NULL                                    NULL                                    2023-04-22 10:42:16.4594 +0000 +0000    0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:16.4594 +0000 +0000    0  NULL  NULL
        	            	858815648560414721  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 10:42:16.499192 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:16.499192 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:16.499192 +0000 +0000  0  NULL  NULL
        	            	858815648691978241  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-22 10:42:16.539344 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:16.539344 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:16.539344 +0000 +0000  0  NULL  NULL
        	            	858815648827277313  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:42:16.580624 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:16.580624 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:16.580624 +0000 +0000  0  NULL  NULL
        	            	858815648956252161  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:42:16.619991 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:16.619991 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:16.619991 +0000 +0000  0  NULL  NULL
        	            	858815649086734337  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-22 10:42:16.659815 +0000 +0000  NULL                                    NULL                                    2023-04-22 10:42:16.659815 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-22 10:42:16.659815 +0000 +0000  0  NULL  NULL
        	            	858815649873690625  AUTO CREATE STATS                Table statistics refresh for system.public.role_members                                 CREATE STATISTICS __auto__ FROM [23] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  running           NULL  2023-04-22 10:42:16.899943 +0000 +0000  2023-04-22 10:42:17.008408 +0000 +0000  NULL                                    2023-04-22 10:42:16.974535 +0000 +0000  0  NULL    1     16291948909908000    2023-04-22 10:42:17.008413 +0000 +0000  2023-04-23 10:42:17.008413 +0000 +0000  1  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (9.52s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@knz knz added the GA-blocker label Apr 23, 2023
@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ 516376912e30ab5911074b07bc86c18e83a0303d:

        	            	859085504175276033  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-23 09:34:49.897336 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:49.897336 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:49.897336 +0000 +0000  0  NULL  NULL
        	            	859085504489029633  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-23 09:34:49.993016 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:49.993016 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:49.993016 +0000 +0000  0  NULL  NULL
        	            	859085504729579521  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 09:34:50.066425 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:50.066425 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:50.066425 +0000 +0000  0  NULL  NULL
        	            	859085504962396161  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 09:34:50.137493 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:50.137493 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:50.137493 +0000 +0000  0  NULL  NULL
        	            	859085505203240961  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 09:34:50.21098 +0000 +0000   NULL                                    NULL                                    2023-04-23 09:34:50.21098 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:50.21098 +0000 +0000   0  NULL  NULL
        	            	859085505464270849  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-23 09:34:50.290648 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:50.290648 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:50.290648 +0000 +0000  0  NULL  NULL
        	            	859085505699381249  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-23 09:34:50.362386 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:50.362386 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:50.362386 +0000 +0000  0  NULL  NULL
        	            	859085505926725633  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-23 09:34:50.431764 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:50.431764 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:50.431764 +0000 +0000  0  NULL  NULL
        	            	859085506157248513  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 09:34:50.502122 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:50.502122 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:50.502122 +0000 +0000  0  NULL  NULL
        	            	859085506386100225  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 09:34:50.57196 +0000 +0000   NULL                                    NULL                                    2023-04-23 09:34:50.57196 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:50.57196 +0000 +0000   0  NULL  NULL
        	            	859085506620456961  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 09:34:50.643475 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:50.643475 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:50.643475 +0000 +0000  0  NULL  NULL
        	            	859085506848620545  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 09:34:50.713106 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:50.713106 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:50.713106 +0000 +0000  0  NULL  NULL
        	            	859085507101589505  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 09:34:50.790309 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:50.790309 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:50.790309 +0000 +0000  0  NULL  NULL
        	            	859085507335749633  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 09:34:50.861759 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:50.861759 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:50.861759 +0000 +0000  0  NULL  NULL
        	            	859085507572727809  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 09:34:50.934089 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:50.934089 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:50.934089 +0000 +0000  0  NULL  NULL
        	            	859085507797876737  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 09:34:51.002798 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:51.002798 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:51.002798 +0000 +0000  0  NULL  NULL
        	            	859085508036919297  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 09:34:51.07575 +0000 +0000   NULL                                    NULL                                    2023-04-23 09:34:51.07575 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:51.07575 +0000 +0000   0  NULL  NULL
        	            	859085508271439873  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:34:51.147316 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:51.147316 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:51.147316 +0000 +0000  0  NULL  NULL
        	            	859085508507435009  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:34:51.219335 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:51.219335 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:51.219335 +0000 +0000  0  NULL  NULL
        	            	859085508543447041  AUTO CREATE STATS                Table statistics refresh for system.public.statement_diagnostics_requests               CREATE STATISTICS __auto__ FROM [35] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-23 09:34:51.230318 +0000 +0000  2023-04-23 09:34:51.402243 +0000 +0000  2023-04-23 09:34:52.217681 +0000 +0000  2023-04-23 09:34:52.217738 +0000 +0000  1  NULL    1     1009461171859846676  2023-04-23 09:34:51.402259 +0000 +0000  2023-04-24 09:34:51.402259 +0000 +0000  1  NULL  NULL
        	            	859085508745986049  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:34:51.292144 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:51.292144 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:51.292144 +0000 +0000  0  NULL  NULL
        	            	859085509049188353  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:34:51.384681 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:54.463153 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:51.384681 +0000 +0000  0  NULL  NULL
        	            	859085509282922497  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:34:51.456011 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:54.543124 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:51.456011 +0000 +0000  0  NULL  NULL
        	            	859085509528092673  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:34:51.530815 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:54.625049 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:51.530815 +0000 +0000  0  NULL  NULL
        	            	859085509764448257  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:34:51.602947 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:55.022117 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:51.602947 +0000 +0000  0  NULL  NULL
        	            	859085510005030913  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:34:51.676372 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:55.116307 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:51.676372 +0000 +0000  0  NULL  NULL
        	            	859085510231097345  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:34:51.745362 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:55.207046 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:51.745362 +0000 +0000  0  NULL  NULL
        	            	859085510474137601  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-23 09:34:51.819523 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:51.819523 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:51.819523 +0000 +0000  0  NULL  NULL
        	            	859085510763118593  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-23 09:34:51.907716 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:51.907716 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:51.907716 +0000 +0000  0  NULL  NULL
        	            	859085511030112257  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-23 09:34:51.989197 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:51.989197 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:51.989197 +0000 +0000  0  NULL  NULL
        	            	859085511286489089  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 09:34:52.067436 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:52.067436 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:52.067436 +0000 +0000  0  NULL  NULL
        	            	859085511534542849  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 09:34:52.143146 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:52.143146 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:52.143146 +0000 +0000  0  NULL  NULL
        	            	859085511773585409  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 09:34:52.216088 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:52.216088 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:52.216088 +0000 +0000  0  NULL  NULL
        	            	859085512013938689  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:34:52.289436 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:54.755277 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:52.289436 +0000 +0000  0  NULL  NULL
        	            	859085512260026369  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:34:52.364553 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:54.842239 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:52.364553 +0000 +0000  0  NULL  NULL
        	            	859085512509784065  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:34:52.440759 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:54.927946 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:52.440759 +0000 +0000  0  NULL  NULL
        	            	859085512759214081  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 09:34:52.516893 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:52.516893 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:52.516893 +0000 +0000  0  NULL  NULL
        	            	859085512995831809  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 09:34:52.589089 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:52.589089 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:52.589089 +0000 +0000  0  NULL  NULL
        	            	859085513002975233  AUTO CREATE STATS                Table statistics refresh for system.public.database_role_settings                       CREATE STATISTICS __auto__ FROM [44] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-23 09:34:52.591251 +0000 +0000  2023-04-23 09:34:52.752355 +0000 +0000  2023-04-23 09:34:53.436627 +0000 +0000  2023-04-23 09:34:53.436666 +0000 +0000  1  NULL    1     1563314763081215743  2023-04-23 09:34:52.752377 +0000 +0000  2023-04-24 09:34:52.752377 +0000 +0000  1  NULL  NULL
        	            	859085513248276481  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 09:34:52.666129 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:52.666129 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:52.666129 +0000 +0000  0  NULL  NULL
        	            	859085513516679169  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 09:34:52.748052 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:52.748052 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:52.748052 +0000 +0000  0  NULL  NULL
        	            	859085513749987329  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 09:34:52.819244 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:52.819244 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:52.819244 +0000 +0000  0  NULL  NULL
        	            	859085513983885313  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 09:34:52.890624 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:52.890624 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:52.890624 +0000 +0000  0  NULL  NULL
        	            	859085514207756289  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 09:34:52.95894 +0000 +0000   NULL                                    NULL                                    2023-04-23 09:34:52.95894 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:52.95894 +0000 +0000   0  NULL  NULL
        	            	859085514441949185  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 09:34:53.030414 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:53.030414 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:53.030414 +0000 +0000  0  NULL  NULL
        	            	859085514691543041  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 09:34:53.106582 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:53.106582 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:53.106582 +0000 +0000  0  NULL  NULL
        	            	859085514931404801  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:34:53.179779 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:53.179779 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:53.179779 +0000 +0000  0  NULL  NULL
        	            	859085515177132033  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:34:53.254772 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:53.254772 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:53.254772 +0000 +0000  0  NULL  NULL
        	            	859085515412963329  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:34:53.326735 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:34:53.326735 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:34:53.326735 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (10.32s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

jobs.TestJobControlByType failed with artifacts on release-23.1.0 @ 516376912e30ab5911074b07bc86c18e83a0303d:

        	            	859086250609967105  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-23 09:38:37.691071 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:37.691071 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:37.691071 +0000 +0000  0  NULL  NULL
        	            	859086250764468225  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-23 09:38:37.738221 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:37.738221 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:37.738221 +0000 +0000  0  NULL  NULL
        	            	859086250920280065  BACKUP                           fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-23 09:38:37.785771 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:37.785771 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:37.785771 +0000 +0000  0  NULL  NULL
        	            	859086251088478209  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 09:38:37.837103 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:37.837103 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:37.837103 +0000 +0000  0  NULL  NULL
        	            	859086251230724097  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 09:38:37.880518 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:37.880518 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:37.880518 +0000 +0000  0  NULL  NULL
        	            	859086251371888641  BACKUP                           fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 09:38:37.923592 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:37.923592 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:37.923592 +0000 +0000  0  NULL  NULL
        	            	859086251513511937  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-23 09:38:37.966804 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:37.966804 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:37.966804 +0000 +0000  0  NULL  NULL
        	            	859086251655004161  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-23 09:38:38.009991 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:38.009991 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.009991 +0000 +0000  0  NULL  NULL
        	            	859086251798953985  BACKUP                           fake job                                                                                                                                                                           testuser  {}  reverting         NULL  2023-04-23 09:38:38.053923 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:38.053923 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.053923 +0000 +0000  0  NULL  NULL
        	            	859086251871666177  AUTO CREATE STATS                Table statistics refresh for system.public.replication_constraint_stats                 CREATE STATISTICS __auto__ FROM [25] WITH OPTIONS THROTTLING 0.9 AS OF SYSTEM TIME '-30s'  root      {}  succeeded         NULL  2023-04-23 09:38:38.076083 +0000 +0000  2023-04-23 09:38:38.18505 +0000 +0000   2023-04-23 09:38:40.070562 +0000 +0000  2023-04-23 09:38:40.070682 +0000 +0000  1  NULL    1     6780067671446147527  2023-04-23 09:38:38.185055 +0000 +0000  2023-04-24 09:38:38.185055 +0000 +0000  1  NULL  NULL
        	            	859086251959681025  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 09:38:38.102959 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:38.102959 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.102959 +0000 +0000  0  NULL  NULL
        	            	859086252133384193  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 09:38:38.155985 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:38.155985 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.155985 +0000 +0000  0  NULL  NULL
        	            	859086252290572289  BACKUP                           fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 09:38:38.203944 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:38.203944 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.203944 +0000 +0000  0  NULL  NULL
        	            	859086252477415425  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 09:38:38.26097 +0000 +0000   NULL                                    NULL                                    2023-04-23 09:38:38.26097 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.26097 +0000 +0000   0  NULL  NULL
        	            	859086252653477889  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 09:38:38.314689 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:38.314689 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.314689 +0000 +0000  0  NULL  NULL
        	            	859086252807061505  BACKUP                           fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 09:38:38.361568 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:38.361568 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.361568 +0000 +0000  0  NULL  NULL
        	            	859086252983812097  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 09:38:38.415509 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:38.415509 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.415509 +0000 +0000  0  NULL  NULL
        	            	859086253154205697  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 09:38:38.467503 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:38.467503 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.467503 +0000 +0000  0  NULL  NULL
        	            	859086253311754241  BACKUP                           fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 09:38:38.515584 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:38.515584 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.515584 +0000 +0000  0  NULL  NULL
        	            	859086253460357121  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:38:38.560939 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:38.560939 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.560939 +0000 +0000  0  NULL  NULL
        	            	859086253619871745  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:38:38.609621 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:38.609621 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.609621 +0000 +0000  0  NULL  NULL
        	            	859086253769261057  BACKUP                           fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:38:38.655207 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:38.655207 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.655207 +0000 +0000  0  NULL  NULL
        	            	859086253936279553  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:38:38.706189 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:42.637569 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.706189 +0000 +0000  0  NULL  NULL
        	            	859086254113849345  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:38:38.760362 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:42.71968 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.760362 +0000 +0000  0  NULL  NULL
        	            	859086254256193537  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:38:38.803841 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:42.801402 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.803841 +0000 +0000  0  NULL  NULL
        	            	859086254437105665  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:38:38.859027 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:43.849802 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.859027 +0000 +0000  0  NULL  NULL
        	            	859086254586822657  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:38:38.904703 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:43.933167 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.904703 +0000 +0000  0  NULL  NULL
        	            	859086254719238145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:38:38.945131 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:44.017211 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.945131 +0000 +0000  0  NULL  NULL
        	            	859086254874918913  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-23 09:38:38.992627 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:38.992627 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:38.992627 +0000 +0000  0  NULL  NULL
        	            	859086255020343297  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-23 09:38:39.037016 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:39.037016 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.037016 +0000 +0000  0  NULL  NULL
        	            	859086255180185601  RESTORE                          fake job                                                                                                                                                                           testuser  {}  paused            NULL  2023-04-23 09:38:39.085789 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:39.085789 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.085789 +0000 +0000  0  NULL  NULL
        	            	859086255334916097  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 09:38:39.13301 +0000 +0000   NULL                                    NULL                                    2023-04-23 09:38:39.13301 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.13301 +0000 +0000   0  NULL  NULL
        	            	859086255501737985  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 09:38:39.183917 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:39.183917 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.183917 +0000 +0000  0  NULL  NULL
        	            	859086255638806529  RESTORE                          fake job                                                                                                                                                                           testuser  {}  failed            NULL  2023-04-23 09:38:39.225756 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:39.225756 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.225756 +0000 +0000  0  NULL  NULL
        	            	859086255787212801  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:38:39.271037 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:43.350171 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.271037 +0000 +0000  0  NULL  NULL
        	            	859086255934013441  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:38:39.315842 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:43.429236 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.315842 +0000 +0000  0  NULL  NULL
        	            	859086256066953217  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:38:39.356407 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:43.761106 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.356407 +0000 +0000  0  NULL  NULL
        	            	859086256200548353  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 09:38:39.397179 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:39.397179 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.397179 +0000 +0000  0  NULL  NULL
        	            	859086256344301569  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 09:38:39.441047 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:39.441047 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.441047 +0000 +0000  0  NULL  NULL
        	            	859086256493592577  RESTORE                          fake job                                                                                                                                                                           testuser  {}  succeeded         NULL  2023-04-23 09:38:39.486607 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:39.486607 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.486607 +0000 +0000  0  NULL  NULL
        	            	859086256647208961  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 09:38:39.533488 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:39.533488 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.533488 +0000 +0000  0  NULL  NULL
        	            	859086256790863873  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 09:38:39.577329 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:39.577329 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.577329 +0000 +0000  0  NULL  NULL
        	            	859086256935174145  RESTORE                          fake job                                                                                                                                                                           testuser  {}  canceled          NULL  2023-04-23 09:38:39.621361 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:39.621361 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.621361 +0000 +0000  0  NULL  NULL
        	            	859086257093509121  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 09:38:39.669696 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:39.669696 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.669696 +0000 +0000  0  NULL  NULL
        	            	859086257233821697  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 09:38:39.712506 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:39.712506 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.712506 +0000 +0000  0  NULL  NULL
        	            	859086257371152385  RESTORE                          fake job                                                                                                                                                                           testuser  {}  cancel-requested  NULL  2023-04-23 09:38:39.75442 +0000 +0000   NULL                                    NULL                                    2023-04-23 09:38:39.75442 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.75442 +0000 +0000   0  NULL  NULL
        	            	859086257573036033  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:38:39.81602 +0000 +0000   NULL                                    NULL                                    2023-04-23 09:38:39.81602 +0000 +0000   0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.81602 +0000 +0000   0  NULL  NULL
        	            	859086257723211777  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:38:39.861868 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:39.861868 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.861868 +0000 +0000  0  NULL  NULL
        	            	859086257871388673  RESTORE                          fake job                                                                                                                                                                           testuser  {}  pause-requested   NULL  2023-04-23 09:38:39.907076 +0000 +0000  NULL                                    NULL                                    2023-04-23 09:38:39.907076 +0000 +0000  0  NULL    NULL  0                    NULL                                    2023-04-23 09:38:39.907076 +0000 +0000  0  NULL  NULL
    --- FAIL: TestJobControlByType/pause_ALL_RESTORE_JOBS (10.92s)

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@yuzefovich
Copy link
Member

fixed by #101858

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-test-failure Broken test (automatically or manually discovered). GA-blocker O-robot Originated from a bot. T-jobs
Projects
None yet
Development

No branches or pull requests

3 participants