Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ccl/schemachangerccl: TestBackupRollbacks_ccl_create_index failed #116740

Closed
cockroach-teamcity opened this issue Dec 19, 2023 · 3 comments
Closed

ccl/schemachangerccl: TestBackupRollbacks_ccl_create_index failed #116740

cockroach-teamcity opened this issue Dec 19, 2023 · 3 comments
Assignees
@rafiss
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. P-1 Issues/test failures with a fix SLA of 1 month T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Milestone
24.1

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Dec 19, 2023 •
edited by cockroach-jira-scripts
Loading

ccl/schemachangerccl.TestBackupRollbacks_ccl_create_index failed on master @ 5f76d93b2ad8d29c14c9b176753ec33cbf81e3fe:

    datadriven.go:144: 
        /var/lib/engflow/worker/work/2/exec/bazel-out/k8-fastbuild/bin/pkg/ccl/schemachangerccl/schemachangerccl_test_/schemachangerccl_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/schemachangerccl/testdata/end_to_end/create_index/create_index.definition:16:
        stage-query [2 args]
        SELECT count(*) = $successfulStageCount*3 FROM t WHERE name = 'a';
        ----
        true
    datadriven.go:144: 
        /var/lib/engflow/worker/work/2/exec/bazel-out/k8-fastbuild/bin/pkg/ccl/schemachangerccl/schemachangerccl_test_/schemachangerccl_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/schemachangerccl/testdata/end_to_end/create_index/create_index.definition:22:
        stage-exec [2 args]
        INSERT INTO t VALUES ($stageKey, 'a');
        INSERT INTO t VALUES ($stageKey + 1, 'b');
        INSERT INTO t VALUES ($stageKey + 2, 'c');
        DELETE FROM t WHERE name = 'a' and id = $stageKey;
        INSERT INTO t VALUES ($stageKey, 'a');
        UPDATE t SET name = 'a' WHERE id > 0;
        ----
    datadriven.go:144: 
        /var/lib/engflow/worker/work/2/exec/bazel-out/k8-fastbuild/bin/pkg/ccl/schemachangerccl/schemachangerccl_test_/schemachangerccl_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/schemachangerccl/testdata/end_to_end/create_index/create_index.definition:33:
        stage-query [2 args]
        SELECT count(*) = $successfulStageCount*3 FROM t WHERE name = 'a';
        ----
        true

pkg/sql/schemachanger/sctest/framework.go:909: (areStmtsFullySupportedAtClusterVersion.func1)
	NOTICE: .ClusterSettings() called via implicit interface ApplicationLayerInterface;
HINT: consider using .ApplicationLayer().ClusterSettings() instead.
TIP: consider replacing the test server initialization from:
    ts, ... := serverutils.StartServer(t, ...)
    defer ts.Stopper().Stop(...)
to:
    srv, ... := serverutils.StartServer(t, ...)
    defer srv.Stopper().Stop(...)
    ts := srv.ApplicationLayer()

See also: https://go.crdb.dev/p/testserver-and-cluster-virtualization
    datadriven.go:144: 
        /var/lib/engflow/worker/work/2/exec/bazel-out/k8-fastbuild/bin/pkg/ccl/schemachangerccl/schemachangerccl_test_/schemachangerccl_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/schemachangerccl/testdata/end_to_end/create_index/create_index.definition:38:
        test [0 args]
        CREATE INDEX idx
          ON defaultdb.t (id, name)
          STORING (money)
          PARTITION BY LIST (id) (PARTITION p1 VALUES IN (1))
        ----
    --- FAIL: TestBackupRollbacks_ccl_create_index/group (11.89s)
=== RUN   TestBackupRollbacks_ccl_create_index/group/post_commit_stage_6_of_7
=== PAUSE TestBackupRollbacks_ccl_create_index/group/post_commit_stage_6_of_7
=== CONT  TestBackupRollbacks_ccl_create_index/group/post_commit_stage_6_of_7
        --- FAIL: TestBackupRollbacks_ccl_create_index/group/post_commit_stage_6_of_7 (115.86s)
=== RUN   TestBackupRollbacks_ccl_create_index/group/post_commit_stage_6_of_7/post_1_rollback_stage_exec
            --- FAIL: TestBackupRollbacks_ccl_create_index/group/post_commit_stage_6_of_7/post_1_rollback_stage_exec (40.00s)

Parameters:

  • attempt=1
  • run=9
  • shard=46
Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/sql-foundations

This test on roachdash | Improve this report!

Jira issue: CRDB-34743
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) labels Dec 19, 2023
@cockroach-teamcity cockroach-teamcity added this to the 24.1 milestone Dec 19, 2023
@cockroach-teamcity
Copy link
Member Author

ccl/schemachangerccl.TestBackupRollbacks_ccl_create_index failed on master @ 53e4efd389167110d687e5dea8bf915fe2582a25:

        UPDATE t SET name = 'a' WHERE id > 0;
        ----
    datadriven.go:144: 
        /var/lib/engflow/worker/work/1/exec/bazel-out/k8-fastbuild/bin/pkg/ccl/schemachangerccl/schemachangerccl_test_/schemachangerccl_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/schemachangerccl/testdata/end_to_end/create_index/create_index.definition:33:
        stage-query [2 args]
        SELECT count(*) = $successfulStageCount*3 FROM t WHERE name = 'a';
        ----
        true

pkg/sql/schemachanger/sctest/framework.go:911: (areStmtsFullySupportedAtClusterVersion.func1)
	NOTICE: .ClusterSettings() called via implicit interface ApplicationLayerInterface;
HINT: consider using .ApplicationLayer().ClusterSettings() instead.
TIP: consider replacing the test server initialization from:
    ts, ... := serverutils.StartServer(t, ...)
    defer ts.Stopper().Stop(...)
to:
    srv, ... := serverutils.StartServer(t, ...)
    defer srv.Stopper().Stop(...)
    ts := srv.ApplicationLayer()

See also: https://go.crdb.dev/p/testserver-and-cluster-virtualization
    datadriven.go:144: 
        /var/lib/engflow/worker/work/1/exec/bazel-out/k8-fastbuild/bin/pkg/ccl/schemachangerccl/schemachangerccl_test_/schemachangerccl_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/schemachangerccl/testdata/end_to_end/create_index/create_index.definition:38:
        test [0 args]
        CREATE INDEX idx
          ON defaultdb.t (id, name)
          STORING (money)
          PARTITION BY LIST (id) (PARTITION p1 VALUES IN (1))
        ----
    --- FAIL: TestBackupRollbacks_ccl_create_index/group (26.94s)
=== RUN   TestBackupRollbacks_ccl_create_index/group/post_commit_stage_1_of_7/post_1_rollback_stage_exec/restore_all_tables_in_database
    backup.go:437: post-RESTORE schema change was expected to not succeed
                --- FAIL: TestBackupRollbacks_ccl_create_index/group/post_commit_stage_1_of_7/post_1_rollback_stage_exec/restore_all_tables_in_database (10.70s)
=== RUN   TestBackupRollbacks_ccl_create_index/group/post_commit_stage_1_of_7/post_2_rollback_stages_exec
            --- FAIL: TestBackupRollbacks_ccl_create_index/group/post_commit_stage_1_of_7/post_2_rollback_stages_exec (30.98s)
=== RUN   TestBackupRollbacks_ccl_create_index/group/post_commit_stage_1_of_7/post_2_rollback_stages_exec/restore_all_tables_in_database
    backup.go:437: post-RESTORE schema change was expected to not succeed
                --- FAIL: TestBackupRollbacks_ccl_create_index/group/post_commit_stage_1_of_7/post_2_rollback_stages_exec/restore_all_tables_in_database (12.22s)
=== RUN   TestBackupRollbacks_ccl_create_index/group/post_commit_stage_1_of_7/post_2_rollback_stages_exec/restore_database
    backup.go:437: post-RESTORE schema change was expected to not succeed
                --- FAIL: TestBackupRollbacks_ccl_create_index/group/post_commit_stage_1_of_7/post_2_rollback_stages_exec/restore_database (9.15s)
=== RUN   TestBackupRollbacks_ccl_create_index/group/post_commit_stage_1_of_7
=== PAUSE TestBackupRollbacks_ccl_create_index/group/post_commit_stage_1_of_7
=== CONT  TestBackupRollbacks_ccl_create_index/group/post_commit_stage_1_of_7
        --- FAIL: TestBackupRollbacks_ccl_create_index/group/post_commit_stage_1_of_7 (124.33s)
=== RUN   TestBackupRollbacks_ccl_create_index/group/post_commit_stage_1_of_7/post_1_rollback_stage_exec
            --- FAIL: TestBackupRollbacks_ccl_create_index/group/post_commit_stage_1_of_7/post_1_rollback_stage_exec (39.70s)
=== RUN   TestBackupRollbacks_ccl_create_index/group/post_commit_stage_1_of_7/post_1_rollback_stage_exec/restore_database_with_schema_only
    backup.go:437: post-RESTORE schema change was expected to not succeed
                --- FAIL: TestBackupRollbacks_ccl_create_index/group/post_commit_stage_1_of_7/post_1_rollback_stage_exec/restore_database_with_schema_only (12.28s)

Parameters:

  • attempt=1
  • run=4
  • shard=46
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@exalate-issue-sync exalate-issue-sync bot added the P-1 Issues/test failures with a fix SLA of 1 month label Dec 23, 2023
@rafiss rafiss removed the release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. label Dec 23, 2023
@cockroach-teamcity
Copy link
Member Author

ccl/schemachangerccl.TestBackupRollbacks_ccl_create_index failed on master @ 57bd0a92676b1c58ee7e29979ce3dceb5837abdb:

    datadriven.go:144: 
        /var/lib/engflow/worker/work/3/exec/bazel-out/k8-fastbuild/bin/pkg/ccl/schemachangerccl/schemachangerccl_test_/schemachangerccl_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/schemachangerccl/testdata/end_to_end/create_index/create_index.definition:16:
        stage-query [2 args]
        SELECT count(*) = $successfulStageCount*3 FROM t WHERE name = 'a';
        ----
        true
    datadriven.go:144: 
        /var/lib/engflow/worker/work/3/exec/bazel-out/k8-fastbuild/bin/pkg/ccl/schemachangerccl/schemachangerccl_test_/schemachangerccl_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/schemachangerccl/testdata/end_to_end/create_index/create_index.definition:22:
        stage-exec [2 args]
        INSERT INTO t VALUES ($stageKey, 'a');
        INSERT INTO t VALUES ($stageKey + 1, 'b');
        INSERT INTO t VALUES ($stageKey + 2, 'c');
        DELETE FROM t WHERE name = 'a' and id = $stageKey;
        INSERT INTO t VALUES ($stageKey, 'a');
        UPDATE t SET name = 'a' WHERE id > 0;
        ----
    datadriven.go:144: 
        /var/lib/engflow/worker/work/3/exec/bazel-out/k8-fastbuild/bin/pkg/ccl/schemachangerccl/schemachangerccl_test_/schemachangerccl_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/schemachangerccl/testdata/end_to_end/create_index/create_index.definition:33:
        stage-query [2 args]
        SELECT count(*) = $successfulStageCount*3 FROM t WHERE name = 'a';
        ----
        true

pkg/sql/schemachanger/sctest/framework.go:911: (areStmtsFullySupportedAtClusterVersion.func1)
	NOTICE: .ClusterSettings() called via implicit interface ApplicationLayerInterface;
HINT: consider using .ApplicationLayer().ClusterSettings() instead.
TIP: consider replacing the test server initialization from:
    ts, ... := serverutils.StartServer(t, ...)
    defer ts.Stopper().Stop(...)
to:
    srv, ... := serverutils.StartServer(t, ...)
    defer srv.Stopper().Stop(...)
    ts := srv.ApplicationLayer()

See also: https://go.crdb.dev/p/testserver-and-cluster-virtualization
    datadriven.go:144: 
        /var/lib/engflow/worker/work/3/exec/bazel-out/k8-fastbuild/bin/pkg/ccl/schemachangerccl/schemachangerccl_test_/schemachangerccl_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/schemachangerccl/testdata/end_to_end/create_index/create_index.definition:38:
        test [0 args]
        CREATE INDEX idx
          ON defaultdb.t (id, name)
          STORING (money)
          PARTITION BY LIST (id) (PARTITION p1 VALUES IN (1))
        ----
    --- FAIL: TestBackupRollbacks_ccl_create_index/group (16.62s)
=== RUN   TestBackupRollbacks_ccl_create_index/group/post_commit_stage_6_of_7
=== PAUSE TestBackupRollbacks_ccl_create_index/group/post_commit_stage_6_of_7
=== CONT  TestBackupRollbacks_ccl_create_index/group/post_commit_stage_6_of_7
        --- FAIL: TestBackupRollbacks_ccl_create_index/group/post_commit_stage_6_of_7 (124.90s)
=== RUN   TestBackupRollbacks_ccl_create_index/group/post_commit_stage_6_of_7/post_1_rollback_stage_exec
            --- FAIL: TestBackupRollbacks_ccl_create_index/group/post_commit_stage_6_of_7/post_1_rollback_stage_exec (37.54s)

Parameters:

  • attempt=1
  • run=16
  • shard=46
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@rafiss rafiss self-assigned this Dec 28, 2023
@rafiss rafiss mentioned this issue Dec 28, 2023
Merged
craig bot pushed a commit that referenced this issue Jan 2, 2024
@erikgrinaker @nvanbenschoten @rafiss
Merge #116319 #117014 #117138
e218b13 
116319: kvnemesis: add Raft application assertions r=erikgrinaker a=erikgrinaker

This adds basic kvnemesis assertions for Raft command application:

* A request is only applied once, at a single log index and lease applied index across all replicas (i.e. no double-applies or replays).

* Commands do not regress the Raft index/term, lease applied index, and closed timestamp (assuming no node restarts).

* All replicas apply the same commands in the same order at the same positions.

Resolves #115771.
Touches #114421.
Epic: none
Release note: None

117014: kv: add mutex tracing to tscache r=erikgrinaker a=nvanbenschoten

We added the ability to trace mutex acquisition in afb73d2. This commit uses this capability to trace the mutex acquisition the timestamp cache.

Epic: None
Release note: None

117138: sctest: remove t.Parallel calls and cleanup r=rafiss a=rafiss

In 943a7fd and f397c13 these tests were made parallel. This makes debugging harder, and now that the tests run in a remote execution environment, the speed benefits seem dubious. Let's make it non-parallel to see if the tests become more stable.

informs #116740
informs #117103
Release note: None

Co-authored-by: Erik Grinaker <grinaker@cockroachlabs.com>
Co-authored-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
Co-authored-by: Rafi Shamim <rafi@cockroachlabs.com>
@rafiss
Copy link
Collaborator

rafiss commented Jan 3, 2024

addressed by #117138

@rafiss rafiss closed this as completed Jan 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rafiss rafiss

Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. P-1 Issues/test failures with a fix SLA of 1 month T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Projects
No open projects
SQL Foundations
Archived in project
Milestone
24.1
Development

No branches or pull requests
2 participants
@rafiss @cockroach-teamcity