pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed

[cockroach-teamcity]

pkg/ccl/testccl/workload/schemachange/schemachange_test.TestWorkload failed with artifacts on master @ f334c73953bf5d5bed7a2c0effd72391284458f8:

     true,
     true,
     true,
     true,
     true
    ]
   }
  ],
  "previousStatements": [
   "ALTER TABLE schema_w1_58.table_w1_69 ALTER PRIMARY KEY USING COLUMNS (\"col\"\"69_w1_72\", \"'col69_w1_73\", col69_w1_75)"
  ]
 }
}
    schema_change_external_test.go:134: 
        	Error Trace:	pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:134
        	Error:      	Received unexpected error:
        	            	***UNEXPECTED ERROR; Received an unexpected execution error.: ERROR: unimplemented: the following columns are not indexable due to their type: col"69_w1_72 (type refcursor), col69_w1_75 (type tsvector) (SQLSTATE 0A000)
        	            	(1) forced error mark
        	            	  | "fatal error when running txn"
        	            	  | github.com/cockroachdb/errors/withstack/*withstack.withStack::
        	            	Wraps: (2)
        	            	Wraps: (3) attached stack trace
        	            	  -- stack trace:
        	            	  | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*opStmt).executeStmt
        	            	  | 	github.com/cockroachdb/cockroach/pkg/workload/schemachange/operation_generator.go:3124
        	            	  | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*schemaChangeWorker).runInTxn
        	            	  | 	github.com/cockroachdb/cockroach/pkg/workload/schemachange/schemachange.go:500
        	            	  | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*schemaChangeWorker).run
        	            	  | 	github.com/cockroachdb/cockroach/pkg/workload/schemachange/schemachange.go:588
        	            	  | pkg/ccl/testccl/workload/schemachange/schemachange_test_test.TestWorkload.TestWorkload.func4.func6
        	            	  | 	pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:123
        	            	  | golang.org/x/sync/errgroup.(*Group).Go.func1
        	            	  | 	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78
        	            	  | runtime.goexit
        	            	  | 	src/runtime/asm_arm64.s:1222
        	            	Wraps: (4) ***UNEXPECTED ERROR; Received an unexpected execution error.
        	            	Wraps: (5) ERROR: unimplemented: the following columns are not indexable due to their type: col"69_w1_72 (type refcursor), col69_w1_75 (type tsvector) (SQLSTATE 0A000)
        	            	Error types: (1) *markers.withMark (2) *schemachange.ErrorState (3) *withstack.withStack (4) *errutil.withPrefix (5) *pgconn.PgError
        	Test:       	TestWorkload
Schema Workload Stats
Total Schema Statements Executed = 96
Total Schema Statements Succeeded = 96
Total Schema Statement Expected Failures = 0
Total Transactions Committed = 54
Total Transactions Rolled Back = 95
Total Transactions Executed = 149
    schema_change_external_test.go:99: backup, tracing data, and system table dumps in /artifacts/tmp/_tmp/67537770b84c3a4ea177539c1e46bd43/logTestWorkload3789403017
    panic.go:626: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/67537770b84c3a4ea177539c1e46bd43/logTestWorkload3789403017
--- FAIL: TestWorkload (13.20s)

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

• pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed #128345 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [C-test-failure O-robot P-2 T-sql-foundations branch-release-24.2]

/cc @cockroachdb/sql-foundations

This test on roachdash | Improve this report!

Jira issue: CRDB-41107

[annrpom]

First step is for me to check why

cockroach/pkg/workload/schemachange/operation_generator.go

Lines 2503 to 2508 in 06e3262

	indexableFamilies := []string{
	"DecimalFamily",
	"IntFamily",
	"StringFamily",
	"UuidFamily",
	}

isn't working

[github-actions]

pkg/ccl/testccl/workload/schemachange/schemachange_test.TestWorkload failed on master @ 769d76a90001dc97dc6d1a9c73ecde1e5242ce9e:

    "queryArgs": [
     "schema_w0_50.table_w1_129",
     "schema_w0_50.table_w1_129"
    ],
    "result": true
   }
  ],
  "previousStatements": [
   "ALTER TABLE schema_w0_50.table_w1_129 ALTER PRIMARY KEY USING COLUMNS (col129_w1_134, \"col12%q9_w1_138\") USING HASH",
   "ALTER TABLE schema_w0_50.table_w1_129 ADD CONSTRAINT \"table_w1_129_co%b5l129_w1_141_table_w1_129_co%b5l129_w1_141_fk\" FOREIGN KEY (\"co%b5l129_w1_141\") REFERENCES schema_w0_50.table_w1_129 (\"co%b5l129_w1_141\") ON DELETE CASCADE ON UPDATE CASCADE"
  ]
 }
}
    schema_change_external_test.go:134: 
        	Error Trace:	pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:134
        	Error:      	Received unexpected error:
        	            	***UNEXPECTED ERROR; Received an unexpected execution error.: ERROR: relation "table_w1_129" (133): unimplemented: cannot perform other schema changes in the same transaction as a primary key change (SQLSTATE 0A000)
        	            	(1) forced error mark
        	            	  | "fatal error when running txn"
        	            	  | github.com/cockroachdb/errors/withstack/*withstack.withStack::
        	            	Wraps: (2)
        	            	Wraps: (3) attached stack trace
        	            	  -- stack trace:
        	            	  | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*opStmt).executeStmt
        	            	  | 	github.com/cockroachdb/cockroach/pkg/workload/schemachange/operation_generator.go:3124
        	            	  | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*schemaChangeWorker).runInTxn
        	            	  | 	github.com/cockroachdb/cockroach/pkg/workload/schemachange/schemachange.go:500
        	            	  | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*schemaChangeWorker).run
        	            	  | 	github.com/cockroachdb/cockroach/pkg/workload/schemachange/schemachange.go:588
        	            	  | pkg/ccl/testccl/workload/schemachange/schemachange_test_test.TestWorkload.TestWorkload.func4.func6
        	            	  | 	pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:123
        	            	  | golang.org/x/sync/errgroup.(*Group).Go.func1
        	            	  | 	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78
        	            	  | runtime.goexit
        	            	  | 	src/runtime/asm_amd64.s:1695
        	            	Wraps: (4) ***UNEXPECTED ERROR; Received an unexpected execution error.
        	            	Wraps: (5) ERROR: relation "table_w1_129" (133): unimplemented: cannot perform other schema changes in the same transaction as a primary key change (SQLSTATE 0A000)
        	            	Error types: (1) *markers.withMark (2) *schemachange.ErrorState (3) *withstack.withStack (4) *errutil.withPrefix (5) *pgconn.PgError
        	Test:       	TestWorkload
Schema Workload Stats
Total Schema Statements Executed = 72
Total Schema Statements Succeeded = 72
Total Schema Statement Expected Failures = 0
Total Transactions Committed = 35
Total Transactions Rolled Back = 59
Total Transactions Executed = 94
    schema_change_external_test.go:99: backup, tracing data, and system table dumps in /var/lib/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/testlogs/pkg/ccl/testccl/workload/schemachange/schemachange_test/test.outputs/logTestWorkload3777156638
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestWorkload3777156638
--- FAIL: TestWorkload (19.85s)

Parameters:

• attempt=1
• run=1
• shard=1

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

• pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed #128345 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [C-test-failure O-robot P-2 T-sql-foundations branch-release-24.2]

This test on roachdash | Improve this report!

[cockroach-teamcity]

pkg/ccl/testccl/workload/schemachange/schemachange_test.TestWorkload failed on master @ 2558f3f3a1dab0907db3dafd510ec5bc2464073f:

   {
    "query": "SELECT EXISTS (SELECT * FROM public.table_w1_155)",
    "result": false
   },
   {
    "query": "WITH descriptors AS (SELECT crdb_internal.pb_to_json('cockroach.sql.sqlbase.Descriptor', descriptor)-\u003e'table' AS d FROM system.descriptor WHERE id = $1::REGCLASS) SELECT EXISTS (SELECT mut FROM (SELECT json_array_elements(d-\u003e'mutations') AS mut FROM descriptors) WHERE (mut-\u003e'primaryKeySwap') IS NOT NULL)",
    "queryArgs": [
     "public.table_w1_155"
    ],
    "result": false
   }
  ],
  "previousStatements": [
   "ALTER TABLE public.table_w1_155 ADD COLUMN col_w1_155_w1_187 FLOAT4 NOT NULL"
  ]
 }
}
    schema_change_external_test.go:134: 
        	Error Trace:	pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:134
        	Error:      	Received unexpected error:
        	            	***UNEXPECTED COMMIT ERROR; Received an unexpected commit error: ERROR: transaction committed but schema change aborted with error: (23502): validation of column "col_w1_155_w1_187" NOT NULL failed on row: col155_w1_156='1979-11-11', col155_w1_157='17a5205f-70ee-43ac-9ad6-62d4df1cfefd', col155_w1_158=e'4\r D"', "col155%q_w1_159"='d2fd8938-979e-4e77-b316-140ccd3b4539', "co&l155_w1_160"=ARRAY['9942:7d7b:c6dd:bbd6:a873:d7b8:985:53d6/24','153.111.230.245/14','199.200.189.17','d9bb:b729:2143:3f33:a8b3:cbf7:7b05:e7a8/16'], "co%pl155_w1_161"=e'\r\x19\x14\x1aq', "col1\\U000D0BBE55_w1_162"=18, col̓155_w1_163='00000000-0000-0000-0000-000000000000', col_w1_155_w0_116=NULL, col_w1_155_w1_187=NULL (SQLSTATE 23502)
        	            	(1) forced error mark
        	            	  | "fatal error when running txn"
        	            	  | github.com/cockroachdb/errors/withstack/*withstack.withStack::
        	            	Wraps: (2)
        	            	Wraps: (3) attached stack trace
        	            	  -- stack trace:
        	            	  | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*schemaChangeWorker).run
        	            	  | 	github.com/cockroachdb/cockroach/pkg/workload/schemachange/schemachange.go:659
        	            	  | pkg/ccl/testccl/workload/schemachange/schemachange_test_test.TestWorkload.TestWorkload.func4.func6
        	            	  | 	pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:123
        	            	  | golang.org/x/sync/errgroup.(*Group).Go.func1
        	            	  | 	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78
        	            	  | runtime.goexit
        	            	  | 	src/runtime/asm_amd64.s:1695
        	            	Wraps: (4) ***UNEXPECTED COMMIT ERROR; Received an unexpected commit error
        	            	Wraps: (5) ERROR: transaction committed but schema change aborted with error: (23502): validation of column "col_w1_155_w1_187" NOT NULL failed on row: col155_w1_156='1979-11-11', col155_w1_157='17a5205f-70ee-43ac-9ad6-62d4df1cfefd', col155_w1_158=e'4\r D"', "col155%q_w1_159"='d2fd8938-979e-4e77-b316-140ccd3b4539', "co&l155_w1_160"=ARRAY['9942:7d7b:c6dd:bbd6:a873:d7b8:985:53d6/24','153.111.230.245/14','199.200.189.17','d9bb:b729:2143:3f33:a8b3:cbf7:7b05:e7a8/16'], "co%pl155_w1_161"=e'\r\x19\x14\x1aq', "col1\\U000D0BBE55_w1_162"=18, col̓155_w1_163='00000000-0000-0000-0000-000000000000', col_w1_155_w0_116=NULL, col_w1_155_w1_187=NULL (SQLSTATE 23502)
        	            	Error types: (1) *markers.withMark (2) *schemachange.ErrorState (3) *withstack.withStack (4) *errutil.withPrefix (5) *pgconn.PgError
        	Test:       	TestWorkload
Schema Workload Stats
Total Schema Statements Executed = 97
Total Schema Statements Succeeded = 97
Total Schema Statement Expected Failures = 2
Total Transactions Committed = 61
Total Transactions Rolled Back = 91
Total Transactions Executed = 152
    schema_change_external_test.go:99: backup, tracing data, and system table dumps in /var/lib/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/testlogs/pkg/ccl/testccl/workload/schemachange/schemachange_test/run_3_of_25/test.outputs/logTestWorkload3863471377
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestWorkload3863471377
--- FAIL: TestWorkload (27.28s)

Parameters:

• attempt=1
• run=3
• shard=1

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

• pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed #128345 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [C-test-failure O-robot P-2 T-sql-foundations branch-release-24.2]

This test on roachdash | Improve this report!

[cockroach-teamcity]

pkg/ccl/testccl/workload/schemachange/schemachange_test.TestWorkload failed with artifacts on master @ c64fb80786b82b0a848bf59e271d8b2ce44db674:

   {
    "query": "SELECT EXISTS (SELECT attrelid::REGCLASS AS table_name, attname AS column_name, pg_get_expr(adbin, adrelid) AS computed_formula FROM pg_attribute JOIN pg_attrdef ON (attrelid = adrelid) AND (attnum = adnum) WHERE (atthasdef AND (attrelid = $1::REGCLASS)) AND (pg_get_expr(adbin, adrelid) ILIKE (('%%' || $2) || '%%')))",
    "queryArgs": [
     "public.table_w1_4",
     "col1😽_w0_9"
    ],
    "result": false
   }
  ],
  "previousStatements": [
   "ALTER TABLE public.table_w1_4 DROP COLUMN col1😽_w0_9"
  ]
 }
}
    schema_change_external_test.go:134: 
        	Error Trace:	pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:134
        	Error:      	Received unexpected error:
        	            	***FAIL; Failed to receive an execution error when errors were expected
        	            	(1) forced error mark
        	            	  | "fatal error when running txn"
        	            	  | github.com/cockroachdb/errors/withstack/*withstack.withStack::
        	            	Wraps: (2)
        	            	Wraps: (3) attached stack trace
        	            	  -- stack trace:
        	            	  | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*opStmt).executeStmt
        	            	  | 	github.com/cockroachdb/cockroach/pkg/workload/schemachange/operation_generator.go:3139
        	            	  | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*schemaChangeWorker).runInTxn
        	            	  | 	github.com/cockroachdb/cockroach/pkg/workload/schemachange/schemachange.go:500
        	            	  | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*schemaChangeWorker).run
        	            	  | 	github.com/cockroachdb/cockroach/pkg/workload/schemachange/schemachange.go:588
        	            	  | pkg/ccl/testccl/workload/schemachange/schemachange_test_test.TestWorkload.TestWorkload.func4.func6
        	            	  | 	pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:123
        	            	  | golang.org/x/sync/errgroup.(*Group).Go.func1
        	            	  | 	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78
        	            	  | runtime.goexit
        	            	  | 	src/runtime/asm_arm64.s:1222
        	            	Wraps: (4) ***FAIL; Failed to receive an execution error when errors were expected
        	            	Error types: (1) *markers.withMark (2) *schemachange.ErrorState (3) *withstack.withStack (4) *errutil.leafError
        	Test:       	TestWorkload
Schema Workload Stats
Total Schema Statements Executed = 10
Total Schema Statements Succeeded = 10
Total Schema Statement Expected Failures = 0
Total Transactions Committed = 6
Total Transactions Rolled Back = 13
Total Transactions Executed = 19
    schema_change_external_test.go:99: backup, tracing data, and system table dumps in /artifacts/tmp/_tmp/67537770b84c3a4ea177539c1e46bd43/logTestWorkload1905478014
    panic.go:626: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/67537770b84c3a4ea177539c1e46bd43/logTestWorkload1905478014
--- FAIL: TestWorkload (4.58s)

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

• pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed #128345 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [C-test-failure O-robot P-2 T-sql-foundations branch-release-24.2]

This test on roachdash | Improve this report!

[cockroach-teamcity]

pkg/ccl/testccl/workload/schemachange/schemachange_test.TestWorkload failed with artifacts on master @ c64fb80786b82b0a848bf59e271d8b2ce44db674:

    "query": "SELECT EXISTS (SELECT attrelid::REGCLASS AS table_name, attname AS column_name, pg_get_expr(adbin, adrelid) AS computed_formula FROM pg_attribute JOIN pg_attrdef ON (attrelid = adrelid) AND (attnum = adnum) WHERE (atthasdef AND (attrelid = $1::REGCLASS)) AND (pg_get_expr(adbin, adrelid) ILIKE (('%%' || $2) || '%%')))",
    "queryArgs": [
     "schema_w0_5.table_w1_120",
     "col11%p5_w0_124"
    ],
    "result": false
   }
  ],
  "previousStatements": [
   "ALTER TABLE schema_w0_5.table_w1_120 DROP COLUMN \"col11%p5_w0_124\""
  ],
  "usesDeclarativeSchemaChanger": true
 }
}
    schema_change_external_test.go:134: 
        	Error Trace:	pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:134
        	Error:      	Received unexpected error:
        	            	***FAIL; Failed to receive an execution error when errors were expected
        	            	(1) forced error mark
        	            	  | "fatal error when running txn"
        	            	  | github.com/cockroachdb/errors/withstack/*withstack.withStack::
        	            	Wraps: (2)
        	            	Wraps: (3) attached stack trace
        	            	  -- stack trace:
        	            	  | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*opStmt).executeStmt
        	            	  | 	github.com/cockroachdb/cockroach/pkg/workload/schemachange/operation_generator.go:3139
        	            	  | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*schemaChangeWorker).runInTxn
        	            	  | 	github.com/cockroachdb/cockroach/pkg/workload/schemachange/schemachange.go:500
        	            	  | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*schemaChangeWorker).run
        	            	  | 	github.com/cockroachdb/cockroach/pkg/workload/schemachange/schemachange.go:588
        	            	  | pkg/ccl/testccl/workload/schemachange/schemachange_test_test.TestWorkload.TestWorkload.func4.func6
        	            	  | 	pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:123
        	            	  | golang.org/x/sync/errgroup.(*Group).Go.func1
        	            	  | 	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78
        	            	  | runtime.goexit
        	            	  | 	src/runtime/asm_arm64.s:1222
        	            	Wraps: (4) ***FAIL; Failed to receive an execution error when errors were expected
        	            	Error types: (1) *markers.withMark (2) *schemachange.ErrorState (3) *withstack.withStack (4) *errutil.leafError
        	Test:       	TestWorkload
Schema Workload Stats
Total Schema Statements Executed = 116
Total Schema Statements Succeeded = 116
Total Schema Statement Expected Failures = 2
Total Transactions Committed = 60
Total Transactions Rolled Back = 120
Total Transactions Executed = 180
    schema_change_external_test.go:99: backup, tracing data, and system table dumps in /artifacts/tmp/_tmp/67537770b84c3a4ea177539c1e46bd43/logTestWorkload3012296048
    panic.go:626: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/67537770b84c3a4ea177539c1e46bd43/logTestWorkload3012296048
--- FAIL: TestWorkload (14.83s)

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

• pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed #128345 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [C-test-failure O-robot P-2 T-sql-foundations branch-release-24.2]

This test on roachdash | Improve this report!