Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ccl/backupccl: package timed out #50984

Closed
cockroach-teamcity opened this issue Jul 5, 2020 · 40 comments
Closed

ccl/backupccl: package timed out #50984

cockroach-teamcity opened this issue Jul 5, 2020 · 40 comments
Assignees
@pbardea
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.
Milestone
20.2

Comments

@cockroach-teamcity
Copy link
Member

(ccl/backupccl).(unknown) failed on master@3e0de239121813ea4d47873388a2828a66d9edf7:

Slow failing tests:
TestBackupRestoreAppend - 2.31s

Slow passing tests:
TestBackupRestorePartitioned - 21.00s
TestBackupRestoreMultiNodeRemote - 19.72s
TestBackupRestoreMultiNodeLocal - 17.00s
TestBackupRestoreSingleNodeLocal - 4.50s
TestBackupRestoreStatementResult - 1.45s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Jul 5, 2020
@cockroach-teamcity cockroach-teamcity added this to the 20.2 milestone Jul 5, 2020
@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@f61d73fe1e4773372f3800b715143305a3342ee7:

Slow failing tests:
TestBackupRestoreAppend - 0.31s

Slow passing tests:
TestBackupRestoreMultiNodeRemote - 3.63s
TestBackupRestorePartitioned - 2.70s
TestBackupRestoreMultiNodeLocal - 2.29s
TestBackupRestoreSingleNodeLocal - 0.91s
More

Parameters:

  • TAGS=
  • GOFLAGS=-parallel=4
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@176c08c801a028d43d0d6f3f7044af078ff3f4f9:

Slow failing tests:
TestBackupDoesNotHangOnIntent - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1338.16s
TestRestoreAsOfSystemTime - 65.26s
TestBackupRestoreUserDefinedSchemas - 55.09s
TestBackupRestoreResume - 54.71s
TestBackupRestoreUserDefinedTypes - 49.10s
TestClientDisconnect - 46.11s
TestBackupRestoreWithConcurrentWrites - 43.84s
TestBackupRestoreNegativePrimaryKey - 42.76s
TestBackupRestoreTenant - 41.71s
TestBackupRestoreCrossTableReferences - 39.61s
TestBackupRestoreSystemJobsProgress - 35.06s
TestBackupAndRestoreJobDescription - 33.62s
TestConcurrentBackupRestores - 33.22s
TestBackupRestorePartitionedIncremental - 33.03s
TestBackupRestorePartitioned - 30.85s
TestProtectedTimestampsDuringBackup - 30.01s
TestBackupRestoreMultiNodeRemote - 29.38s
TestBackupRestorePartitionedMergeDirectories - 29.05s
TestBackupRestoreDataDriven - 27.68s
TestBackupRestoreSystemTables - 25.39s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@75ccf016099aa089b0dfee9932398862e6bf7d5c:

Slow failing tests:
TestBackupRestoreTenant/restore-tenant10-to-latest - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1340.48s
TestBackupRestoreResume - 76.46s
TestRestoreAsOfSystemTime - 67.79s
TestBackupRestoreUserDefinedSchemas - 60.92s
TestBackupRestoreDataDriven - 52.87s
TestBackupRestoreUserDefinedTypes - 50.44s
TestBackupRestoreNegativePrimaryKey - 40.37s
TestBackupRestoreCrossTableReferences - 39.52s
TestBackupRestorePartitioned - 37.81s
TestConcurrentBackupRestores - 37.02s
TestBackupRestoreWithConcurrentWrites - 34.23s
TestBackupRestorePartitionedMergeDirectories - 30.28s
TestBackupAndRestoreJobDescription - 30.19s
TestBackupRestoreSystemJobsProgress - 30.07s
TestProtectedTimestampsDuringBackup - 29.81s
TestBackupRestoreMultiNodeRemote - 28.35s
TestBackupRestorePartitionedIncremental - 27.33s
TestBackupRestoreMultiNodeLocal - 25.78s
TestBackupRestoreInterleaved - 24.75s
TestBackupRestoreSystemTables - 22.86s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@d438437fe2b9dbec3980f9265a36c9dfb527ba8b:

Slow failing tests:
TestClusterRevisionHistory/t0 - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1223.01s
TestClusterRestoreFailCleanup - 71.01s
TestRestoreAsOfSystemTime - 57.44s
TestBackupRestoreDataDriven - 49.68s
TestBackupRestoreUserDefinedSchemas - 47.74s
TestBackupRestoreResume - 46.24s
TestBackupRestoreUserDefinedTypes - 42.55s
TestClientDisconnect - 40.10s
TestBackupRestoreNegativePrimaryKey - 34.17s
TestBackupRestoreTenant - 33.59s
TestBackupRestoreCrossTableReferences - 33.21s
TestBackupRestoreWithConcurrentWrites - 31.25s
TestBackupRestorePartitioned - 30.89s
TestConcurrentBackupRestores - 29.59s
TestBackupRestoreSystemJobsProgress - 28.90s
TestBackupRestoreMultiNodeRemote - 27.60s
TestBackupRestoreMultiNodeLocal - 25.49s
TestProtectedTimestampsDuringBackup - 25.35s
TestBackupAndRestoreJobDescription - 25.26s
TestBackupRestorePartitionedMergeDirectories - 25.24s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@a688bb7e4e0c1d996131bd9b586eab79cb72973f:

Slow failing tests:
TestProtectedTimestampsDuringBackup - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1420.63s
TestRestoreAsOfSystemTime - 66.83s
TestBackupRestoreUserDefinedSchemas - 56.22s
TestBackupRestoreDataDriven - 54.58s
TestBackupRestoreUserDefinedTypes - 49.94s
TestBackupRestoreResume - 46.31s
TestBackupRestoreCrossTableReferences - 40.77s
TestBackupRestoreNegativePrimaryKey - 37.18s
TestBackupRestoreWithConcurrentWrites - 36.46s
TestBackupRestorePartitionedMergeDirectories - 35.54s
TestBackupAndRestoreJobDescription - 34.80s
TestBackupRestorePartitioned - 33.29s
TestBackupRestorePartitionedIncremental - 31.70s
TestConcurrentBackupRestores - 31.27s
TestBackupRestoreMultiNodeLocal - 29.86s
TestBackupRestoreMultiNodeRemote - 29.84s
TestBackupRestoreSystemJobsProgress - 28.72s
TestBackupRestoreInterleaved - 26.16s
TestBackupRestoreSequenceOwnership - 22.17s
TestBackupRestoreSystemTables - 21.71s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@66a076f89e92becc1d7e1a2ecfd39c275165e2d8:

Slow failing tests:
TestBackupRestoreTenant - 0.01s

Slow passing tests:
TestBackupRestoreAppend - 1362.39s
TestRestoreAsOfSystemTime - 68.19s
TestBackupRestoreUserDefinedSchemas - 56.76s
TestBackupRestoreResume - 55.21s
TestBackupRestoreDataDriven - 51.91s
TestBackupRestoreUserDefinedTypes - 48.29s
TestBackupRestoreNegativePrimaryKey - 46.63s
TestConcurrentBackupRestores - 39.24s
TestBackupRestoreWithConcurrentWrites - 38.98s
TestBackupRestoreCrossTableReferences - 38.30s
TestBackupRestorePartitionedIncremental - 35.40s
TestBackupAndRestoreJobDescription - 31.43s
TestBackupRestorePartitioned - 30.89s
TestProtectedTimestampsDuringBackup - 30.47s
TestBackupRestorePartitionedMergeDirectories - 29.49s
TestBackupRestoreSystemJobsProgress - 29.35s
TestBackupRestoreMultiNodeRemote - 28.59s
TestBackupRestoreMultiNodeLocal - 27.03s
TestBackupRestoreSystemTables - 25.45s
TestBackupRestoreInterleaved - 24.78s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@1be32272030366639be5b4093c6d45bc143978ba:

Slow failing tests:
TestFullClusterBackupDroppedTables - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1318.51s
TestRestoreAsOfSystemTime - 58.22s
TestBackupRestoreUserDefinedSchemas - 50.94s
TestBackupRestoreDataDriven - 49.54s
TestBackupRestoreUserDefinedTypes - 44.76s
TestClientDisconnect - 43.64s
TestBackupRestoreNegativePrimaryKey - 43.19s
TestBackupRestoreResume - 40.56s
TestBackupRestoreCrossTableReferences - 37.36s
TestBackupRestoreWithConcurrentWrites - 35.53s
TestBackupRestoreTenant - 35.29s
TestConcurrentBackupRestores - 35.16s
TestBackupRestorePartitioned - 34.28s
TestProtectedTimestampsDuringBackup - 29.30s
TestBackupRestorePartitionedMergeDirectories - 27.95s
TestBackupRestoreMultiNodeRemote - 27.28s
TestBackupRestoreMultiNodeLocal - 26.45s
TestBackupAndRestoreJobDescription - 26.25s
TestBackupRestorePartitionedIncremental - 25.51s
TestBackupRestoreSystemJobsProgress - 25.11s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@0c12d8b101a853f441926ad758c4c2882ecd33ee:

Slow failing tests:
TestOfflineDescriptorsDuringRestore/restore-into-existing-database - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1307.35s
TestRestoreAsOfSystemTime - 68.36s
TestBackupRestoreResume - 68.32s
TestBackupRestoreDataDriven - 53.30s
TestBackupRestoreUserDefinedSchemas - 52.81s
TestBackupRestoreUserDefinedTypes - 48.54s
TestClientDisconnect - 45.87s
TestBackupRestoreCrossTableReferences - 38.51s
TestBackupRestoreNegativePrimaryKey - 37.05s
TestConcurrentBackupRestores - 36.96s
TestBackupRestoreWithConcurrentWrites - 36.46s
TestBackupRestoreTenant - 35.04s
TestBackupRestorePartitioned - 34.59s
TestBackupRestoreMultiNodeRemote - 29.79s
TestBackupRestoreSystemJobsProgress - 28.10s
TestProtectedTimestampsDuringBackup - 27.97s
TestBackupRestoreMultiNodeLocal - 27.49s
TestBackupAndRestoreJobDescription - 27.42s
TestBackupRestorePartitionedIncremental - 27.12s
TestBackupRestorePartitionedMergeDirectories - 25.88s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@5a7f32fd41cb84d35562db09761912ff9c903700:

Slow failing tests:
TestClusterRestoreFailCleanup - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1245.10s
TestRestoreAsOfSystemTime - 63.17s
TestBackupRestoreUserDefinedSchemas - 52.75s
TestBackupRestoreResume - 52.13s
TestBackupRestoreDataDriven - 49.72s
TestBackupRestoreUserDefinedTypes - 45.03s
TestClientDisconnect - 42.89s
TestBackupRestoreCrossTableReferences - 38.08s
TestBackupRestoreTenant - 35.01s
TestBackupRestoreWithConcurrentWrites - 34.87s
TestBackupRestoreNegativePrimaryKey - 33.69s
TestConcurrentBackupRestores - 32.66s
TestBackupRestoreSystemJobsProgress - 29.28s
TestBackupRestoreMultiNodeRemote - 28.12s
TestBackupRestorePartitioned - 27.98s
TestBackupAndRestoreJobDescription - 27.77s
TestBackupRestorePartitionedMergeDirectories - 26.27s
TestBackupRestoreMultiNodeLocal - 25.32s
TestBackupRestorePartitionedIncremental - 25.22s
TestProtectedTimestampsDuringBackup - 24.88s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@0cbfe9be4311af7facc0307ba4e4246680760635:

Slow failing tests:
TestClientDisconnect/BACKUP - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1236.42s
TestBackupRestoreDataDriven - 70.69s
TestBackupRestoreNegativePrimaryKey - 70.08s
TestRestoreAsOfSystemTime - 65.44s
TestBackupRestoreUserDefinedSchemas - 57.31s
TestBackupRestoreResume - 56.56s
TestBackupRestoreUserDefinedTypes - 51.34s
TestBackupRestoreCrossTableReferences - 47.08s
TestBackupRestoreTenant - 40.26s
TestBackupRestorePartitioned - 36.72s
TestBackupRestorePartitionedMergeDirectories - 35.52s
TestBackupAndRestoreJobDescription - 35.51s
TestBackupRestoreWithConcurrentWrites - 34.72s
TestConcurrentBackupRestores - 34.14s
TestProtectedTimestampsDuringBackup - 33.79s
TestBackupRestoreMultiNodeLocal - 33.62s
TestBackupRestoreMultiNodeRemote - 33.61s
TestBackupRestoreSystemTables - 30.87s
TestBackupRestoreSystemJobsProgress - 28.74s
TestBackupRestoreInterleaved - 28.63s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@6fa7a0948a0f5cd5b77098b898ad3706360461bc:

Slow failing tests:
TestBackupRestoreUserDefinedSchemas/multi-schemas - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1828.16s
TestBackupRestoreDataDriven - 74.22s
TestBackupRestoreResume - 59.77s
TestBackupRestoreNegativePrimaryKey - 58.86s
TestBackupRestorePartitioned - 42.37s
TestBackupAndRestoreJobDescription - 40.56s
TestBackupRestorePartitionedMergeDirectories - 40.32s
TestBackupRestoreMultiNodeRemote - 35.50s
TestBackupRestoreSystemJobsProgress - 32.35s
TestBackupRestoreMultiNodeLocal - 31.37s
TestBackupRestoreSystemJobs - 25.10s
TestBackupRestoreSystemTables - 24.78s
TestBackupRestoreSingleUserfile - 14.99s
TestBackupRestoreSingleNodeLocal - 7.35s
TestRestoreFailDatabaseCleanup - 7.24s
TestBackupRestoreEmpty - 6.87s
TestRestoreFailCleanup - 6.75s
TestRestoreFailCleansUpTypeBackReferences - 6.26s
TestBackupRestoreStatementResult - 2.54s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@602aa34aca935b5dc17e2cde6c71b672866ec770:

Slow failing tests:
TestConcurrentBackupRestores - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1613.85s
TestBackupRestoreDataDriven - 70.00s
TestBackupRestoreUserDefinedSchemas - 58.57s
TestBackupRestoreUserDefinedTypes - 54.84s
TestBackupRestoreCrossTableReferences - 50.94s
TestBackupRestoreResume - 50.52s
TestBackupRestoreNegativePrimaryKey - 49.83s
TestBackupRestoreSystemTables - 38.10s
TestBackupRestorePartitionedIncremental - 37.35s
TestBackupAndRestoreJobDescription - 36.64s
TestBackupRestorePartitioned - 35.23s
TestBackupRestoreWithConcurrentWrites - 33.50s
TestBackupRestorePartitionedMergeDirectories - 31.73s
TestBackupRestoreMultiNodeRemote - 30.32s
TestBackupRestoreMultiNodeLocal - 29.04s
TestBackupRestoreInterleaved - 28.66s
TestBackupRestoreSystemJobsProgress - 28.24s
TestBackupRestoreSystemJobs - 28.18s
TestBackupRestoreIncremental - 14.12s
TestBackupRestoreSingleUserfile - 13.32s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@203415533cc0400e6d2e0095aed3453f3cad7cb7:

Slow failing tests:
TestDetachedBackup - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1452.86s
TestBackupRestoreResume - 76.81s
TestBackupRestoreDataDriven - 65.44s
TestRestoreAsOfSystemTime - 63.60s
TestBackupRestoreUserDefinedSchemas - 54.00s
TestBackupRestoreUserDefinedTypes - 47.86s
TestConcurrentBackupRestores - 47.56s
TestBackupRestoreNegativePrimaryKey - 43.78s
TestBackupRestoreWithConcurrentWrites - 42.13s
TestBackupRestoreCrossTableReferences - 39.07s
TestBackupRestorePartitioned - 31.42s
TestBackupAndRestoreJobDescription - 31.22s
TestBackupRestorePartitionedMergeDirectories - 30.93s
TestBackupRestoreSystemJobsProgress - 30.62s
TestBackupRestorePartitionedIncremental - 30.56s
TestBackupRestoreMultiNodeRemote - 29.25s
TestBackupRestoreInterleaved - 26.85s
TestBackupRestoreMultiNodeLocal - 25.84s
TestBackupRestoreSystemTables - 25.25s
TestBackupRestoreSystemJobs - 24.68s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@272c28032bb21fc9000258954fff95f42169ead0:

Slow failing tests:
TestClientDisconnect/BACKUP - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1425.46s
TestBackupRestoreDataDriven - 60.60s
TestRestoreAsOfSystemTime - 58.97s
TestBackupRestoreUserDefinedSchemas - 51.48s
TestBackupRestoreResume - 44.94s
TestBackupRestoreUserDefinedTypes - 44.81s
TestBackupRestoreWithConcurrentWrites - 38.73s
TestBackupRestoreTenant - 36.67s
TestBackupRestoreNegativePrimaryKey - 36.22s
TestBackupRestoreCrossTableReferences - 35.73s
TestBackupRestoreSystemJobsProgress - 32.08s
TestBackupRestorePartitioned - 32.05s
TestConcurrentBackupRestores - 31.49s
TestBackupRestorePartitionedIncremental - 30.19s
TestBackupAndRestoreJobDescription - 29.14s
TestProtectedTimestampsDuringBackup - 29.06s
TestBackupRestorePartitionedMergeDirectories - 27.43s
TestBackupRestoreMultiNodeRemote - 26.13s
TestBackupRestoreMultiNodeLocal - 23.85s
TestBackupRestoreInterleaved - 23.23s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@6cab8b33c6f1c4f37e660cfe8149aaff32b4fce2:

Slow failing tests:
TestRestoreOldVersions/v2.1.8 - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1052.65s
TestRestoreMidSchemaChange - 94.82s
TestClusterRestoreFailCleanup - 74.65s
TestBackupRestoreDataDriven - 61.13s
TestRestoreAsOfSystemTime - 55.72s
TestBackupRestoreUserDefinedSchemas - 48.83s
TestBackupRestoreUserDefinedTypes - 43.73s
TestClusterRevisionHistory - 42.45s
TestBackupRestoreNegativePrimaryKey - 38.69s
TestBackupRestoreResume - 38.67s
TestBackupRestoreCrossTableReferences - 37.91s
TestClientDisconnect - 37.90s
TestBackupRestoreTenant - 34.71s
TestConcurrentBackupRestores - 29.81s
TestProtectedTimestampsDuringBackup - 29.18s
TestBackupRestoreWithConcurrentWrites - 29.01s
TestBackupRestorePartitioned - 28.91s
TestBackupRestorePartitionedIncremental - 28.91s
TestBackupAndRestoreJobDescription - 27.83s
TestBackupRestoreMultiNodeRemote - 26.33s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@7f02a7ac0949f226d617955337965f681706bace:

Slow failing tests:
TestCreateBackupScheduleCollectionOverwrite - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1331.95s
TestRestoreAsOfSystemTime - 63.47s
TestBackupRestoreDataDriven - 62.42s
TestBackupRestoreUserDefinedSchemas - 60.20s
TestBackupRestoreResume - 49.01s
TestBackupRestoreUserDefinedTypes - 46.84s
TestClientDisconnect - 42.31s
TestBackupRestoreNegativePrimaryKey - 38.81s
TestBackupRestoreCrossTableReferences - 37.51s
TestBackupRestoreTenant - 36.82s
TestConcurrentBackupRestores - 33.63s
TestBackupRestoreWithConcurrentWrites - 32.28s
TestBackupAndRestoreJobDescription - 29.36s
TestBackupRestorePartitionedMergeDirectories - 28.93s
TestBackupRestorePartitioned - 28.86s
TestBackupRestoreMultiNodeRemote - 27.30s
TestBackupRestorePartitionedIncremental - 26.71s
TestBackupRestoreSystemJobsProgress - 26.68s
TestProtectedTimestampsDuringBackup - 26.51s
TestBackupRestoreMultiNodeLocal - 24.27s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@d816ce972c4d28fcde15925ebc04e7c3522dbf20:

Slow failing tests:
TestRestoreAsOfSystemTime/latest - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1574.60s
TestBackupRestoreDataDriven - 63.15s
TestBackupRestoreUserDefinedSchemas - 58.68s
TestBackupRestoreResume - 56.40s
TestBackupRestoreUserDefinedTypes - 44.19s
TestBackupRestoreNegativePrimaryKey - 36.88s
TestBackupRestoreCrossTableReferences - 35.99s
TestBackupRestorePartitioned - 33.83s
TestConcurrentBackupRestores - 31.11s
TestBackupRestoreWithConcurrentWrites - 29.57s
TestBackupRestoreMultiNodeRemote - 28.53s
TestBackupAndRestoreJobDescription - 27.89s
TestBackupRestoreSystemJobsProgress - 26.41s
TestBackupRestoreMultiNodeLocal - 26.30s
TestBackupRestorePartitionedMergeDirectories - 25.85s
TestBackupRestorePartitionedIncremental - 24.62s
TestBackupRestoreSystemJobs - 21.51s
TestBackupRestoreInterleaved - 21.42s
TestBackupRestoreSystemTables - 21.07s
TestBackupRestoreSingleUserfile - 12.27s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@9b97bda17396aabc4519b9fa272c39dfe5e851da:

Slow failing tests:
TestAsOfSystemTimeOnRestoredData - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1102.67s
TestRestoreAsOfSystemTime - 327.72s
TestBackupRestoreCrossTableReferences - 217.35s
TestBackupRestoreDataDriven - 61.18s
TestBackupRestoreUserDefinedSchemas - 49.54s
TestBackupRestoreResume - 48.57s
TestBackupRestoreUserDefinedTypes - 44.85s
TestConcurrentBackupRestores - 38.42s
TestBackupRestoreNegativePrimaryKey - 37.12s
TestBackupRestoreWithConcurrentWrites - 35.31s
TestBackupRestorePartitioned - 28.58s
TestBackupAndRestoreJobDescription - 28.08s
TestBackupRestorePartitionedMergeDirectories - 27.80s
TestBackupRestoreSystemJobsProgress - 27.55s
TestBackupRestoreMultiNodeRemote - 26.25s
TestBackupRestoreMultiNodeLocal - 25.04s
TestBackupRestorePartitionedIncremental - 24.98s
TestBackupRestoreSystemTables - 22.90s
TestBackupRestoreSystemJobs - 20.81s
TestBackupRestoreInterleaved - 20.12s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@ea42b57cbd348864b7d389cb1cf3cd70fff61bfa:

Slow failing tests:
TestBackupRestoreUserDefinedSchemas/full-cluster - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1538.50s
TestBackupRestoreDataDriven - 56.85s
TestBackupRestoreResume - 39.13s
TestBackupRestoreNegativePrimaryKey - 33.30s
TestBackupRestorePartitioned - 28.41s
TestBackupAndRestoreJobDescription - 26.57s
TestBackupRestoreMultiNodeRemote - 25.63s
TestBackupRestorePartitionedMergeDirectories - 25.07s
TestBackupRestoreMultiNodeLocal - 24.34s
TestBackupRestoreSystemJobsProgress - 24.06s
TestBackupRestoreSystemTables - 20.00s
TestBackupRestoreSystemJobs - 18.76s
TestBackupRestoreSingleUserfile - 11.20s
TestBackupRestoreSingleNodeLocal - 5.92s
TestBackupRestoreEmpty - 5.56s
TestRestoreFailDatabaseCleanup - 5.46s
TestRestoreFailCleansUpTypeBackReferences - 5.08s
TestRestoreFailCleanup - 4.67s
TestBackupRestoreStatementResult - 2.07s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@05826e3a748d3bf7634130d49667627e687d5875:

Slow failing tests:
TestBackupRestoreUserDefinedSchemas/full-cluster - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1648.67s
TestBackupRestoreDataDriven - 62.28s
TestBackupRestoreResume - 40.36s
TestBackupRestoreNegativePrimaryKey - 33.28s
TestBackupAndRestoreJobDescription - 32.87s
TestBackupRestorePartitioned - 29.90s
TestBackupRestorePartitionedMergeDirectories - 27.86s
TestBackupRestoreMultiNodeRemote - 27.56s
TestBackupRestoreMultiNodeLocal - 27.32s
TestBackupRestoreSystemJobsProgress - 26.08s
TestBackupRestoreSystemJobs - 19.92s
TestBackupRestoreSystemTables - 19.75s
TestBackupRestoreSingleUserfile - 12.11s
TestBackupRestoreSingleNodeLocal - 6.35s
TestRestoreFailDatabaseCleanup - 6.29s
TestBackupRestoreEmpty - 5.58s
TestRestoreFailCleanup - 5.04s
TestRestoreFailCleansUpTypeBackReferences - 4.78s
TestBackupRestoreStatementResult - 2.24s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@47044feed11ec0c0390989bf8f44e777ec3eb00d:

Slow failing tests:
TestBackupRestoreUserDefinedSchemas/remap - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1509.96s
TestBackupRestoreDataDriven - 60.93s
TestBackupRestoreResume - 41.74s
TestBackupRestoreNegativePrimaryKey - 36.43s
TestBackupRestoreSystemJobsProgress - 30.96s
TestBackupAndRestoreJobDescription - 30.46s
TestBackupRestorePartitioned - 29.67s
TestBackupRestoreMultiNodeRemote - 27.55s
TestBackupRestorePartitionedMergeDirectories - 25.87s
TestBackupRestoreSystemJobs - 25.59s
TestBackupRestoreMultiNodeLocal - 25.19s
TestBackupRestoreSystemTables - 22.92s
TestBackupRestoreSingleUserfile - 11.94s
TestBackupRestoreEmpty - 6.08s
TestRestoreFailDatabaseCleanup - 6.00s
TestBackupRestoreSingleNodeLocal - 5.95s
TestRestoreFailCleansUpTypeBackReferences - 5.33s
TestRestoreFailCleanup - 5.33s
TestBackupRestoreStatementResult - 2.18s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@b1abf9c8dfb5880fce69dfc7240e593f077bf77c:

Slow failing tests:
TestRestoreAsOfSystemTime/ts6 - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1219.84s
TestBackupRestoreDataDriven - 63.29s
TestBackupRestoreResume - 55.35s
TestBackupRestoreUserDefinedSchemas - 54.50s
TestBackupRestoreUserDefinedTypes - 48.75s
TestBackupRestoreCrossTableReferences - 44.63s
TestBackupRestoreNegativePrimaryKey - 36.27s
TestBackupRestoreWithConcurrentWrites - 30.30s
TestBackupRestorePartitioned - 29.80s
TestConcurrentBackupRestores - 29.19s
TestBackupAndRestoreJobDescription - 29.12s
TestBackupRestoreMultiNodeRemote - 27.91s
TestBackupRestoreSystemJobsProgress - 27.31s
TestBackupRestorePartitionedMergeDirectories - 26.95s
TestBackupRestoreMultiNodeLocal - 25.86s
TestBackupRestoreInterleaved - 25.07s
TestBackupRestorePartitionedIncremental - 24.53s
TestBackupRestoreSystemJobs - 22.08s
TestBackupRestoreSystemTables - 21.06s
TestBackupRestoreSingleUserfile - 12.05s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@d752fa2bd9afad255e8c655de9c7edc6dad14486:

Slow failing tests:
TestBackupRestoreUserDefinedTypes/revision-history/ts1 - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1466.90s
TestBackupRestoreDataDriven - 63.29s
TestBackupRestoreResume - 52.44s
TestBackupRestoreUserDefinedSchemas - 52.12s
TestBackupRestoreNegativePrimaryKey - 36.28s
TestBackupRestoreSystemJobsProgress - 31.39s
TestBackupRestorePartitioned - 30.82s
TestBackupAndRestoreJobDescription - 27.54s
TestBackupRestoreMultiNodeLocal - 27.28s
TestBackupRestorePartitionedMergeDirectories - 27.19s
TestBackupRestoreMultiNodeRemote - 26.99s
TestBackupRestoreSystemJobs - 20.89s
TestBackupRestoreSystemTables - 20.57s
TestBackupRestoreSingleUserfile - 12.24s
TestBackupRestoreSingleNodeLocal - 6.30s
TestRestoreFailDatabaseCleanup - 5.70s
TestBackupRestoreEmpty - 5.56s
TestRestoreFailCleansUpTypeBackReferences - 4.90s
TestRestoreFailCleanup - 4.81s
TestBackupRestoreStatementResult - 2.06s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@ab503e2fd708541e5e9ebb9a6f2651eda506f2ef:

Slow failing tests:
TestBackupRestoreCrossTableReferences/restore_multi-table_view - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1728.97s
TestBackupRestoreResume - 66.65s
TestBackupRestoreDataDriven - 66.10s
TestBackupRestoreUserDefinedSchemas - 56.81s
TestBackupRestoreUserDefinedTypes - 47.20s
TestBackupRestoreNegativePrimaryKey - 46.40s
TestBackupRestorePartitioned - 35.23s
TestBackupAndRestoreJobDescription - 32.42s
TestBackupRestoreSystemJobsProgress - 30.82s
TestBackupRestorePartitionedMergeDirectories - 29.95s
TestBackupRestoreMultiNodeRemote - 29.21s
TestBackupRestoreMultiNodeLocal - 27.41s
TestBackupRestoreSystemTables - 26.07s
TestBackupRestoreSystemJobs - 25.91s
TestBackupRestoreInterleaved - 25.34s
TestBackupRestoreSingleUserfile - 12.80s
TestRestoreFailDatabaseCleanup - 7.50s
TestBackupRestoreEmpty - 6.90s
TestRestoreFailCleansUpTypeBackReferences - 6.54s
TestBackupRestoreSingleNodeLocal - 6.51s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@1d46df77dbd8721cccf508fb5ed498f3de78022c:

Slow failing tests:
TestBackupRestoreUserDefinedSchemas/full-cluster - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1873.38s
TestBackupRestoreDataDriven - 74.02s
TestBackupRestoreResume - 58.49s
TestBackupRestoreNegativePrimaryKey - 52.63s
TestBackupAndRestoreJobDescription - 41.11s
TestBackupRestorePartitioned - 37.38s
TestBackupRestoreMultiNodeRemote - 36.79s
TestBackupRestorePartitionedMergeDirectories - 36.46s
TestBackupRestoreSystemTables - 31.69s
TestBackupRestoreMultiNodeLocal - 31.62s
TestBackupRestoreSystemJobsProgress - 29.69s
TestBackupRestoreSystemJobs - 23.77s
TestBackupRestoreSingleUserfile - 14.00s
TestBackupRestoreEmpty - 7.77s
TestBackupRestoreSingleNodeLocal - 7.30s
TestRestoreFailDatabaseCleanup - 6.50s
TestRestoreFailCleanup - 5.38s
TestRestoreFailCleansUpTypeBackReferences - 5.12s
TestBackupRestoreStatementResult - 2.51s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@jordanlewis jordanlewis assigned dt and unassigned andreimatei Oct 21, 2020
@jordanlewis
Copy link
Member

@dt, we're seeing timeouts on this package nearly every day. @rytaft will be marking TestBackupRestoreAppend as skipped under race to avoid this issue.

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@40b7942025de0d8e347d25451611ad2c20267d48:

Slow failing tests:
TestBackupRestoreIncremental - 0.00s

Slow passing tests:
TestBackupRestoreAppend - 1653.39s
TestBackupRestoreDataDriven - 61.92s
TestBackupRestoreUserDefinedSchemas - 49.36s
TestBackupRestoreUserDefinedTypes - 43.60s
TestBackupRestoreNegativePrimaryKey - 39.36s
TestBackupRestoreResume - 38.07s
TestBackupRestoreCrossTableReferences - 35.38s
TestBackupRestoreSystemJobsProgress - 30.87s
TestBackupRestorePartitionedMergeDirectories - 30.12s
TestBackupAndRestoreJobDescription - 29.59s
TestBackupRestorePartitioned - 28.88s
TestBackupRestoreMultiNodeRemote - 26.05s
TestBackupRestoreSystemJobs - 26.00s
TestBackupRestoreMultiNodeLocal - 25.52s
TestBackupRestoreInterleaved - 23.67s
TestBackupRestoreSystemTables - 23.67s
TestBackupRestoreSingleUserfile - 11.73s
TestBackupRestoreEmpty - 6.03s
TestRestoreFailDatabaseCleanup - 5.81s
TestBackupRestoreSingleNodeLocal - 5.64s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@jordanlewis
Copy link
Member

@rytaft reminder to please send a skip for TestBackupRestoreAppend.

@jordanlewis
Copy link
Member

Err, a skip-under-race.

@rytaft rytaft mentioned this issue Oct 22, 2020
Merged
@rytaft
Copy link
Collaborator

rytaft commented Oct 22, 2020

Thanks for the reminder - done.

rytaft added a commit to rytaft/cockroach that referenced this issue Oct 22, 2020
@rytaft
ccl/backupccl: skip TestBackupRestorePartitioned under race
f179fd2 
Refs: cockroachdb#50984

Reason: flaky test

Generated by bin/skip-test.

Release justification: non-production code changes

Release note: None
rytaft added a commit to rytaft/cockroach that referenced this issue Oct 22, 2020
@rytaft
ccl/backupccl: skip TestBackupRestoreAppend under race
aa486a6 
Refs: cockroachdb#50984, cockroachdb#54599

Reason: flaky test

Generated by bin/skip-test.

Release justification: non-production code changes

Release note: None
craig bot pushed a commit that referenced this issue Oct 22, 2020
@yuzefovich @rytaft @jordanlewis
Merge #55828 #55853 #55857 #55863
badbaad 
55828: roachtest: disable range merge queue in tpchvec/tpcdsvec r=yuzefovich a=yuzefovich

This commit disables the range merge queue on tpchvec and tpcdsvec
roachtests in order to - hopefully - reduce some noise about the
performance slowness. I'm guessing that some of the variation that we've
been observing with tpchvec/perf tests might come due to range merge
activity, so let's remove that variable. However, I don't have concrete
data to back this guess up.

Release note: None

55853: ccl/backupccl: skip TestBackupRestoreAppend under race r=rytaft a=rytaft

Refs: #50984, #54599

Reason: flaky test

Generated by bin/skip-test.

Release justification: non-production code changes

Release note: None

55857: cmd,skip: add flag to skip test under race r=rytaft a=rytaft

This commit updates the `skip-test` command to support a flag `-under_race`,
which only skips the test under race builds. This commit also adds a new
function to the skip package, `UnderRaceWithIssue`, which makes it easier to
skip a test under race and log the associated issue.

Release note: None

55863: util: add skip.UnderMetamorphic; skip a test r=jordanlewis a=jordanlewis

One test cannot be run under the metamorphic build configuration, since
its output depends on the batch size: the test counts the number of
spans scanned, which will go up with a small batch size due to the table
reader resuming the scan until its finished.

Release note: None

Co-authored-by: Yahor Yuzefovich <yahor@cockroachlabs.com>
Co-authored-by: Rebecca Taft <becca@cockroachlabs.com>
Co-authored-by: Jordan Lewis <jordanthelewis@gmail.com>
@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@75234294d22cc9a6e12724b043f49e78f0ac92d2:

Slow failing tests:
TestScheduleBackupRecoversFromClusterDown - 995.63s
TestBackupRestoreDataDriven - 45.05s
TestBackupRestoreNegativePrimaryKey - 40.35s
TestBackupRestoreMultiNodeRemote - 26.49s
TestBackupRestoreMultiNodeLocal - 24.43s
TestBackupRestoreSingleUserfile - 12.19s
TestBackupRestoreSingleNodeLocal - 6.49s
TestAsOfSystemTimeOnRestoredData - 4.64s
TestClusterRestoreEmptyDB - 0.00s

Slow passing tests:
TestFullClusterBackupDroppedTables - 236.28s
TestBackupRestoreResume - 69.78s
TestRestoreAsOfSystemTime - 57.60s
TestBackupRestoreUserDefinedSchemas - 50.98s
TestBackupRestoreUserDefinedTypes - 44.78s
TestClientDisconnect - 44.33s
TestBackupRestoreWithConcurrentWrites - 37.52s
TestConcurrentBackupRestores - 37.05s
TestBackupRestoreCrossTableReferences - 36.04s
TestBackupRestoreTenant - 34.12s
TestBackupRestorePartitioned - 29.21s
TestProtectedTimestampsDuringBackup - 28.17s
TestBackupRestorePartitionedMergeDirectories - 27.72s
TestBackupRestorePartitionedIncremental - 27.63s
TestBackupRestoreSystemJobsProgress - 27.33s
TestBackupAndRestoreJobDescription - 27.00s
TestBackupRestoreInterleaved - 22.81s
TestFullClusterBackup - 22.71s
TestBackupRestoreSystemTables - 20.48s
TestBackupRestoreSystemJobs - 20.39s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@8aceac3c99c3addece3a9ef9af04cc74715cdb37:

Slow failing tests:
TestScheduleBackupRecoversFromClusterDown - 447.84s
TestDisallowFullClusterRestoreOnNonFreshCluster - 0.00s

Slow passing tests:
TestIncrementalFullClusterBackup - 528.92s
TestFullClusterBackup - 112.24s
TestBackupRestoreDataDriven - 63.26s
TestBackupRestoreResume - 61.69s
TestRestoreAsOfSystemTime - 56.93s
TestClientDisconnect - 49.32s
TestBackupRestoreUserDefinedSchemas - 49.06s
TestBackupRestoreUserDefinedTypes - 43.48s
TestBackupRestoreWithConcurrentWrites - 37.75s
TestConcurrentBackupRestores - 37.10s
TestBackupRestoreCrossTableReferences - 36.31s
TestBackupRestoreNegativePrimaryKey - 36.22s
TestBackupRestoreTenant - 34.00s
TestBackupRestorePartitioned - 29.06s
TestProtectedTimestampsDuringBackup - 28.71s
TestBackupRestorePartitionedIncremental - 28.40s
TestBackupRestorePartitionedMergeDirectories - 27.80s
TestBackupRestoreSystemJobsProgress - 27.76s
TestBackupRestoreMultiNodeRemote - 27.68s
TestBackupAndRestoreJobDescription - 27.60s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@6184870a438ae34afbcf29dda5452345dc7587d3:

Slow failing tests:
TestFullClusterBackup - 0.00s

Slow passing tests:
TestBackupRestoreResume - 82.42s
TestBackupRestoreWithConcurrentWrites - 82.15s
TestBackupRestorePartitionedIncremental - 70.68s
TestBackupRestoreDataDriven - 70.00s
TestBackupRestoreUserDefinedSchemas - 61.84s
TestRestoreAsOfSystemTime - 59.67s
TestBackupRestoreCrossTableReferences - 58.25s
TestBackupRestoreUserDefinedTypes - 58.05s
TestConcurrentBackupRestores - 51.02s
TestBackupRestoreNegativePrimaryKey - 46.14s
TestClientDisconnect - 40.74s
TestBackupAndRestoreJobDescription - 36.65s
TestBackupRestoreSystemJobsProgress - 36.56s
TestBackupRestorePartitioned - 35.62s
TestBackupRestoreInterleaved - 34.85s
TestBackupRestoreMultiNodeRemote - 33.65s
TestBackupRestorePartitionedMergeDirectories - 33.19s
TestBackupRestoreTenant - 31.58s
TestBackupRestoreMultiNodeLocal - 30.33s
TestProtectedTimestampsDuringBackup - 24.95s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@5e3c201595fc33b0d120057c61413195716f811d:

Slow failing tests:
TestIncrementalFullClusterBackup - 0.00s

Slow passing tests:
TestScheduleBackupRecoversFromClusterDown - 560.84s
TestFullClusterBackup - 388.60s
TestBackupRestoreResume - 70.18s
TestBackupRestoreDataDriven - 64.53s
TestRestoreAsOfSystemTime - 56.96s
TestBackupRestoreUserDefinedSchemas - 50.89s
TestClientDisconnect - 46.92s
TestBackupRestoreUserDefinedTypes - 45.71s
TestBackupRestoreNegativePrimaryKey - 39.97s
TestBackupRestoreWithConcurrentWrites - 39.17s
TestBackupRestoreTenant - 38.31s
TestBackupRestoreCrossTableReferences - 37.09s
TestConcurrentBackupRestores - 33.86s
TestProtectedTimestampsDuringBackup - 31.46s
TestBackupRestorePartitionedMergeDirectories - 29.03s
TestBackupRestorePartitioned - 28.92s
TestBackupAndRestoreJobDescription - 28.83s
TestBackupRestoreSystemJobsProgress - 28.25s
TestBackupRestorePartitionedIncremental - 28.02s
TestBackupRestoreMultiNodeRemote - 27.52s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@mwang1026 mwang1026 assigned pbardea and unassigned dt Oct 27, 2020
@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@70fb9a54d6c4d7b3c1d8ad93bb399eefef8f5d04:

Slow failing tests:
TestEmptyFullClusterRestore - 0.00s

Slow passing tests:
TestFullClusterBackupDroppedTables - 496.82s
TestScheduleBackupRecoversFromClusterDown - 278.05s
TestBackupRestoreResume - 75.80s
TestBackupRestoreDataDriven - 67.02s
TestRestoreAsOfSystemTime - 57.43s
TestBackupRestoreUserDefinedSchemas - 51.99s
TestBackupRestoreUserDefinedTypes - 46.62s
TestClientDisconnect - 45.58s
TestBackupRestoreNegativePrimaryKey - 44.47s
TestBackupRestoreWithConcurrentWrites - 39.00s
TestBackupRestoreCrossTableReferences - 37.49s
TestBackupRestoreTenant - 37.19s
TestConcurrentBackupRestores - 36.23s
TestBackupAndRestoreJobDescription - 31.94s
TestBackupRestorePartitioned - 31.59s
TestBackupRestorePartitionedMergeDirectories - 30.37s
TestBackupRestoreSystemJobsProgress - 29.94s
TestBackupRestorePartitionedIncremental - 29.90s
TestProtectedTimestampsDuringBackup - 29.59s
TestBackupRestoreMultiNodeRemote - 28.64s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@157b6dc59552b26fb7ef7968ea3b79577c336423:

Slow failing tests:
TestScheduleBackupRecoversFromClusterDown - 917.67s
TestFullClusterBackupDroppedTables - 0.00s

Slow passing tests:
TestBackupRestoreResume - 73.38s
TestBackupRestoreDataDriven - 67.82s
TestRestoreAsOfSystemTime - 62.74s
TestBackupRestoreUserDefinedSchemas - 53.56s
TestClientDisconnect - 49.29s
TestBackupRestoreUserDefinedTypes - 47.45s
TestBackupRestoreNegativePrimaryKey - 40.45s
TestConcurrentBackupRestores - 39.58s
TestBackupRestoreCrossTableReferences - 38.94s
TestBackupRestoreWithConcurrentWrites - 37.62s
TestBackupRestoreTenant - 36.42s
TestBackupRestoreSystemJobsProgress - 31.76s
TestBackupRestorePartitioned - 31.23s
TestBackupAndRestoreJobDescription - 30.93s
TestFullClusterBackup - 30.82s
TestBackupRestorePartitionedIncremental - 30.48s
TestBackupRestorePartitionedMergeDirectories - 29.72s
TestProtectedTimestampsDuringBackup - 29.64s
TestBackupRestoreMultiNodeLocal - 28.69s
TestBackupRestoreMultiNodeRemote - 28.54s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@8bfb1370ba5bb8c216e88f19e829427e5975f679:

Slow failing tests:
TestFullClusterBackup - 0.00s

Slow passing tests:
TestScheduleBackup - 679.47s
TestScheduleBackupRecoversFromClusterDown - 176.67s
TestBackupRestoreTenant - 120.39s
TestBackupRestoreDataDriven - 63.75s
TestBackupRestoreResume - 61.76s
TestRestoreAsOfSystemTime - 55.91s
TestBackupRestoreUserDefinedSchemas - 51.30s
TestClientDisconnect - 50.88s
TestBackupRestoreUserDefinedTypes - 44.90s
TestBackupRestoreNegativePrimaryKey - 40.78s
TestBackupRestoreWithConcurrentWrites - 39.34s
TestBackupRestoreCrossTableReferences - 37.10s
TestConcurrentBackupRestores - 36.96s
TestProtectedTimestampsDuringBackup - 30.61s
TestBackupRestorePartitioned - 30.27s
TestBackupRestoreMultiNodeRemote - 29.58s
TestBackupAndRestoreJobDescription - 29.56s
TestBackupRestorePartitionedMergeDirectories - 29.43s
TestBackupRestoreSystemJobsProgress - 28.39s
TestBackupRestorePartitionedIncremental - 28.02s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@61c96aaca632dfba5154d55d82c5af0732053b72:

Slow failing tests:
TestEmptyFullClusterRestore - 0.00s

Slow passing tests:
TestFullClusterBackup - 319.06s
TestCreateBackupScheduleRequiresAdminRole - 143.41s
TestBackupRestoreTenant - 129.82s
TestBackupRestoreResume - 76.72s
TestBackupRestoreDataDriven - 67.91s
TestRestoreAsOfSystemTime - 58.61s
TestBackupRestoreUserDefinedSchemas - 53.66s
TestClientDisconnect - 50.92s
TestBackupRestoreUserDefinedTypes - 48.45s
TestConcurrentBackupRestores - 46.99s
TestBackupRestoreWithConcurrentWrites - 44.24s
TestBackupRestoreCrossTableReferences - 42.14s
TestBackupRestoreNegativePrimaryKey - 39.76s
TestBackupRestorePartitioned - 36.51s
TestProtectedTimestampsDuringBackup - 30.31s
TestBackupRestoreMultiNodeRemote - 29.82s
TestBackupRestorePartitionedIncremental - 29.39s
TestBackupAndRestoreJobDescription - 29.12s
TestBackupRestoreSystemJobsProgress - 28.97s
TestBackupRestorePartitionedMergeDirectories - 28.45s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(ccl/backupccl).(unknown) failed on master@f2b53d1080f1ccc503263dd178ee31b4878d2360:

Slow failing tests:
TestFullClusterBackup - 579.57s
TestScheduleBackup - 193.27s
TestClusterRestoreEmptyDB - 0.00s

Slow passing tests:
TestEmptyFullClusterRestore - 295.59s
TestBackupRestoreTenant - 128.53s
TestBackupRestoreResume - 74.77s
TestBackupRestoreDataDriven - 66.82s
TestRestoreAsOfSystemTime - 59.30s
TestBackupRestoreUserDefinedSchemas - 53.23s
TestClientDisconnect - 51.79s
TestBackupRestoreUserDefinedTypes - 47.64s
TestBackupRestoreCrossTableReferences - 43.41s
TestBackupRestoreNegativePrimaryKey - 40.06s
TestConcurrentBackupRestores - 39.47s
TestBackupRestoreWithConcurrentWrites - 38.98s
TestBackupRestorePartitioned - 34.21s
TestBackupRestorePartitionedIncremental - 33.33s
TestProtectedTimestampsDuringBackup - 32.42s
TestBackupRestoreSystemJobsProgress - 29.92s
TestBackupRestoreMultiNodeRemote - 29.76s
TestBackupAndRestoreJobDescription - 29.42s
TestBackupRestorePartitionedMergeDirectories - 29.38s
TestBackupRestoreMultiNodeLocal - 27.70s
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=2
make stressrace TESTS=(unknown) PKG=./pkg/ccl/backupccl TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@pbardea
Copy link
Contributor

pbardea commented Nov 30, 2020

We recently skipped a lot of the slower tests under race. Closing out this issue for now. We are also discussing other approaches to speeding up these tests.

@pbardea pbardea closed this as completed Nov 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pbardea pbardea

Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.
Projects
None yet
Milestone
20.2
Development

No branches or pull requests
6 participants
@dt @jordanlewis @andreimatei @pbardea @rytaft @cockroach-teamcity