sql: TestBackfillWithProtectedTS failed

[cockroach-teamcity]

sql.TestBackfillWithProtectedTS failed on release-25.1 @ 581433a5f096e05cb0654a114120c9f782f06588:

=== RUN   TestBackfillWithProtectedTS
    test_log_scope.go:165: test logs captured to: outputs.zip/logTestBackfillWithProtectedTS2098903094
    test_log_scope.go:76: use -show-logs to present logs inline
    test_server_shim.go:139: cluster virtualization disabled in global scope due to issue: #76378 (expected label: C-bug)
    backfill_protected_timestamp_test.go:526: -- test log scope end --
test logs left over in: outputs.zip/logTestBackfillWithProtectedTS2098903094
--- FAIL: TestBackfillWithProtectedTS (487.93s)
=== RUN   TestBackfillWithProtectedTS/create_index
    backfill_protected_timestamp_test.go:477: running backfill with PTS not setup early enough
    backfill_protected_timestamp_test.go:425: i=0 begin running operations in the middle of backfill
    backfill_protected_timestamp_test.go:429: i=0 finished running operations in the middle of backfill. retErr: failed to DELETE and INSERT: pq: batch timestamp 1737872318.191142743,0 must be after replica GC threshold 1737872338.618148642,0 (r79: /{Table/107/3-Max})
    backfill_protected_timestamp_test.go:474: backfill goroutine failed: expected error was not hit
    backfill_protected_timestamp_test.go:515: 
        	Error Trace:	pkg/sql/backfill_protected_timestamp_test.go:515
        	Error:      	Received unexpected error:
        	            	failed to DELETE and INSERT: pq: batch timestamp 1737872318.191142743,0 must be after replica GC threshold 1737872338.618148642,0 (r79: /{Table/107/3-Max})
        	            	(1) attached stack trace
        	            	  -- stack trace:
        	            	  | github.com/cockroachdb/cockroach/pkg/sql_test.TestBackfillWithProtectedTS.func7.2.1
        	            	  | 	pkg/sql/backfill_protected_timestamp_test.go:438
        	            	  | github.com/cockroachdb/cockroach/pkg/sql_test.TestBackfillWithProtectedTS.func7.2
        	            	  | 	pkg/sql/backfill_protected_timestamp_test.go:463
        	            	  | github.com/cockroachdb/cockroach/pkg/sql_test.TestBackfillWithProtectedTS.func7.Group.GoCtx.4
        	            	  | 	pkg/util/ctxgroup/ctxgroup.go:189
        	            	  | golang.org/x/sync/errgroup.(*Group).Go.func1
        	            	  | 	external/org_golang_x_sync/errgroup/errgroup.go:78
        	            	  | runtime.goexit
        	            	  | 	src/runtime/asm_amd64.s:1695
        	            	Wraps: (2) failed to DELETE and INSERT
        	            	Wraps: (3) pq: batch timestamp 1737872318.191142743,0 must be after replica GC threshold 1737872338.618148642,0 (r79: /{Table/107/3-Max})
        	            	Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *pq.Error
        	Test:       	TestBackfillWithProtectedTS/create_index
    --- FAIL: TestBackfillWithProtectedTS/create_index (436.62s)

Parameters:

• attempt=1
• run=5
• shard=14

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

• sql: TestBackfillWithProtectedTS failed #139829 sql: TestBackfillWithProtectedTS failed [C-test-failure O-robot T-sql-foundations branch-release-24.3.4-rc release-blocker]
• sql: TestBackfillWithProtectedTS failed #139814 sql: TestBackfillWithProtectedTS failed [C-test-failure O-robot T-sql-foundations branch-master release-blocker]

/cc @cockroachdb/sql-foundations

This test on roachdash | Improve this report!

Jira issue: CRDB-46852

[rafiss]

resolved by #139796