Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

roachtest: decommission/nodes=4/duration=1h0m0s failed #98017

Closed
cockroach-teamcity opened this issue Mar 4, 2023 · 3 comments
Closed

roachtest: decommission/nodes=4/duration=1h0m0s failed #98017

cockroach-teamcity opened this issue Mar 4, 2023 · 3 comments
Assignees
Labels
A-kv Anything in KV that doesn't belong in a more specific category. branch-master Failures and bugs on the master branch. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-kv KV Team
Milestone

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Mar 4, 2023

roachtest.decommission/nodes=4/duration=1h0m0s failed with artifacts on master @ fcea283ebca17a6d923c5d4b0401697438b77dbd:

test artifacts and logs in: /artifacts/decommission/nodes=4/duration=1h0m0s/run_1
(decommission.go:361).runDecommission: COMMAND_PROBLEM: ssh verbose log retained in ssh_062513.192435486_n1_cockroach-node-decom.log: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=4 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

/cc @cockroachdb/kv-triage

This test on roachdash | Improve this report!

Jira issue: CRDB-25016

@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. labels Mar 4, 2023
@cockroach-teamcity cockroach-teamcity added this to the 23.1 milestone Mar 4, 2023
@blathers-crl blathers-crl bot added the T-kv KV Team label Mar 4, 2023
@cockroach-teamcity
Copy link
Member Author

roachtest.decommission/nodes=4/duration=1h0m0s failed with artifacts on master @ cf14ad694ee562676f53e36fa8495206c3aed61f:

test artifacts and logs in: /artifacts/decommission/nodes=4/duration=1h0m0s/run_1
(decommission.go:361).runDecommission: COMMAND_PROBLEM: ssh verbose log retained in ssh_062443.476257457_n1_cockroach-node-decom.log: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=4 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.decommission/nodes=4/duration=1h0m0s failed with artifacts on master @ 14b43be03c1c246765be17aa15245493a42c401d:

test artifacts and logs in: /artifacts/decommission/nodes=4/duration=1h0m0s/run_1
(decommission.go:361).runDecommission: COMMAND_PROBLEM: ssh verbose log retained in ssh_062545.224924547_n1_cockroach-node-decom.log: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=4 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

This test on roachdash | Improve this report!

@pav-kv pav-kv added A-kv Anything in KV that doesn't belong in a more specific category. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. labels Mar 6, 2023
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Mar 7, 2023
This change fixes decommission roachtests to properly be aware of the
new output introduced by cockroachdb#96100, and to utilize the decommission
pre-checks accordingly. In some places, the decommission pre-checks are
skipped, especially because the decommission used in the test is not
expected to be completeable.

Fixes: cockroachdb#98026, cockroachdb#98018, cockroachdb#98017.

Release note: None
@cockroach-teamcity
Copy link
Member Author

roachtest.decommission/nodes=4/duration=1h0m0s failed with artifacts on master @ 43a25a09f92408accdeb3cfed1eb35e02d375877:

test artifacts and logs in: /artifacts/decommission/nodes=4/duration=1h0m0s/run_1
(decommission.go:361).runDecommission: COMMAND_PROBLEM: ssh verbose log retained in ssh_062622.766799563_n1_cockroach-node-decom.log: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=4 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

This test on roachdash | Improve this report!

AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Mar 7, 2023
This change fixes decommission roachtests to properly be aware of the
new output introduced by cockroachdb#96100, and to utilize the decommission
pre-checks accordingly. In some places, the decommission pre-checks are
skipped, especially because the decommission used in the test is not
expected to be completeable.

Fixes: cockroachdb#98026, cockroachdb#98018, cockroachdb#98017.

Release note: None
craig bot pushed a commit that referenced this issue Mar 7, 2023
97766: kvflowcontrol: implement kvflowcontrol.Dispatch r=irfansharif a=irfansharif

Part of #95563. Dispatch is a concrete implementation of the kvflowcontrol.Dispatch interface. It's used to dispatch information about admitted raft log entries to the specific nodes where (i) said entries originated, (ii) flow tokens were deducted and (iii) are waiting to be returned. This type is also used to read pending dispatches, which will be used in the raft transport layer when looking to piggyback information on traffic already bound to specific nodes. Since timely dispatching (read: piggybacking) is not guaranteed, we allow querying for all long-overdue dispatches.

Internally it's able to coalesce dispatches bound for a given node. If dispatching admission information for two log entries with the same <RangeID,StoreID,WorkPriority> triple, with log positions L1 and L2 where L1 < L2, we can simply dispatch the one with L2. We leave the integration of this type with the {Store,}WorkQueue (#97599) + raft transport to future PRs.

Release note: None

97826: sql: introduce array_cat_agg aggregate builtin r=yuzefovich a=yuzefovich

This commit introduces a new `array_cat_agg` aggregate builtin function
that takes in an array type as its input, and then unnests each array
and appends all its elements into a single result array. In other
words, it behaves similar to `array_agg(unnest(array_column))`. This
function doesn't have an analogue in Postgres. However, some of our SQL
observability tools need this functionality, and the current workaround
of using a LATERAL JOIN often results in slow apply joins, so this new
builtin should speed things up significantly. In particular,
`crdb_internal.statement_statistics` view is now refactored to use the
new builtin which removes an apply join from it. The choice of this
particular name comes from the fact that we have the `array_cat` builtin
which concatenates two arrays.

Fixes: #97502.

Release note (sql change): New aggregate builtin function
`array_cat_agg` is introduced. It behaves similar to how
`array_agg(unnest(array_column))` would - namely, it takes arrays as its
input, unnests them into the array elements which are then aggregated
into a single result array (i.e. it's similar to concatenating all input
arrays into a single one).

98079: generate-bazel-extra: add instructions to update timeouts list r=rickystewart a=healthy-pod

This code change removes `pkg/ccl/backupccl` from the really enormous targets list because it's only there for CI stress purposes.

It also adds instructions to follow when adding a new test target to the list of really enormous timeouts.

Release note: None
Epic: none

98113: roachtest: decommission roachtests conform to new output r=kvoli a=AlexTalks

This change fixes decommission roachtests to properly be aware of the new output introduced by #96100, and to utilize the decommission pre-checks accordingly. In some places, the decommission pre-checks are skipped, especially because the decommission used in the test is not expected to be completeable.

Fixes: #98026, #98018, #98017.

Release note: None

Co-authored-by: irfan sharif <irfanmahmoudsharif@gmail.com>
Co-authored-by: Yahor Yuzefovich <yahor@cockroachlabs.com>
Co-authored-by: healthy-pod <ahmad@cockroachlabs.com>
Co-authored-by: Alex Sarkesian <sarkesian@cockroachlabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-kv Anything in KV that doesn't belong in a more specific category. branch-master Failures and bugs on the master branch. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-kv KV Team
Projects
None yet
Development

No branches or pull requests

3 participants