Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set mget task claim strategy as the default #197070

Merged

Conversation

mikecote
Copy link
Contributor

@mikecote mikecote commented Oct 21, 2024

Resolves #194625

In this PR, I'm setting mget as the default task claiming strategy along the following changes:

Flaky test runners:

@mikecote mikecote added Feature:Task Manager Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) labels Oct 21, 2024
@mikecote mikecote self-assigned this Oct 21, 2024
@mikecote mikecote added release_note:skip Skip the PR/issue when compiling release notes v9.0.0 backport:prev-minor Backport to (9.0) the previous minor version (i.e. one version back from main) v8.17.0 labels Oct 21, 2024
@kibanamachine
Copy link
Contributor

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#7197

[✅] x-pack/test/alerting_api_integration/observability/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/basic/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config_with_schedule_circuit_breaker.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group4/config_non_dedicated_task_runner.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config.ts: 10/10 tests passed.
[❌] x-pack/test/alerting_api_integration/security_and_spaces/group2/config.ts: 4/10 tests passed.
[❌] x-pack/test/alerting_api_integration/security_and_spaces/group4/config.ts: 2/10 tests passed.
[❌] x-pack/test/alerting_api_integration/security_and_spaces/group2/config_non_dedicated_task_runner.ts: 0/10 tests passed.
[❌] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group3/config.ts: 8/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/actions/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group1/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group1/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group2/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/action_task_params/config.ts: 10/10 tests passed.
[❌] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group4/config.ts: 9/10 tests passed.
[❌] x-pack/test/task_manager_claimer_update_by_query/config.ts: 0/10 tests passed.
[✅] x-pack/test/plugin_api_integration/config.ts: 10/10 tests passed.

see run history

…om:mikecote/kibana into task-manager/mget-default-claim-strategy-2
@kibanamachine
Copy link
Contributor

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#7199

[✅] x-pack/test/alerting_api_integration/observability/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group2/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config_with_schedule_circuit_breaker.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group2/config_non_dedicated_task_runner.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group4/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group4/config_non_dedicated_task_runner.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/actions/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group1/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/basic/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group3/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group2/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/action_task_params/config.ts: 10/10 tests passed.
[❌] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group4/config.ts: 0/10 tests passed.
[❌] x-pack/test/alerting_api_integration/security_and_spaces/group1/config.ts: 0/10 tests passed.
[❌] x-pack/test/task_manager_claimer_update_by_query/config.ts: 4/10 tests passed.
[❌] x-pack/test/plugin_api_integration/config.ts: 9/10 tests passed.

see run history

@kibanamachine
Copy link
Contributor

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#7206

[✅] x-pack/test/alerting_api_integration/security_and_spaces/group1/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group2/config_non_dedicated_task_runner.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/observability/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group4/config_non_dedicated_task_runner.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group4/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config_with_schedule_circuit_breaker.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/action_task_params/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group2/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group2/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group3/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group1/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group4/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/basic/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/actions/config.ts: 10/10 tests passed.
[❌] x-pack/test/task_manager_claimer_update_by_query/config.ts: 1/10 tests passed.
[❌] x-pack/test/plugin_api_integration/config.ts: 0/10 tests passed.

see run history

@kibanamachine
Copy link
Contributor

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7209

[✅] x-pack/test/alerting_api_integration/security_and_spaces/group1/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/observability/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/basic/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group2/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group4/config_non_dedicated_task_runner.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/action_task_params/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group4/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group1/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group2/config_non_dedicated_task_runner.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/actions/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group2/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group3/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config_with_schedule_circuit_breaker.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group4/config.ts: 10/10 tests passed.
[✅] x-pack/test/task_manager_claimer_update_by_query/config.ts: 10/10 tests passed.
[✅] x-pack/test/plugin_api_integration/config.ts: 10/10 tests passed.

see run history

@kibanamachine
Copy link
Contributor

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7224

[✅] x-pack/test/alerting_api_integration/observability/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/basic/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config_with_schedule_circuit_breaker.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group4/config_non_dedicated_task_runner.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group2/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group2/config_non_dedicated_task_runner.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/action_task_params/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/actions/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group4/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group3/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group2/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group1/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group1/config.ts: 10/10 tests passed.
[✅] x-pack/test/alerting_api_integration/spaces_only/tests/alerting/group4/config.ts: 10/10 tests passed.
[✅] x-pack/test/task_manager_claimer_update_by_query/config.ts: 10/10 tests passed.
[✅] x-pack/test/plugin_api_integration/config.ts: 10/10 tests passed.

see run history

@mikecote mikecote marked this pull request as ready for review October 23, 2024 23:33
@mikecote mikecote requested review from a team as code owners October 23, 2024 23:33
@elasticmachine
Copy link
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

@kc13greiner kc13greiner self-requested a review October 24, 2024 12:19
Copy link
Member

@jbudz jbudz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.buildkite/ftr_platform_stateful_configs.yml

Copy link
Contributor

@kc13greiner kc13greiner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Contributor

@ymao1 ymao1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚀

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
taskManager 65 64 -1
Unknown metric groups

API count

id before after diff
taskManager 109 108 -1

History

cc @mikecote

@mikecote mikecote merged commit c31f11e into elastic:main Oct 25, 2024
38 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11518666754

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 25, 2024
Resolves elastic#194625

In this PR, I'm setting `mget` as the default task claiming strategy
along the following changes:
- Given we no longer need the 8.16 specific PRs
(elastic#196317 and
elastic#196757), I've also reverted them.
- Given we now use `met` as the default, I've renamed
`task_manager_claimer_mget` to `task_manager_claimer_update_by_query`
and made tests in that folder test using the `update_by_query` claim
strategy.
- Stabilize flaky tests caused by mget + polling for tasks more
frequently

Flaky test runners:
-
[[59b71bc](elastic@59b71bc)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7197
-
[[aea910e](elastic@aea910e)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7199
-
[[4723ced](elastic@4723ced)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7206
-
[[d28c8c5](elastic@d28c8c5)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7209
-
[[dd7773a](elastic@dd7773a)]
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7224

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit c31f11e)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Oct 25, 2024
# Backport

This will backport the following commits from `main` to `8.x`:
- [Set mget task claim strategy as the default
(#197070)](#197070)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Mike
Côté","email":"mikecote@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-10-25T12:57:46Z","message":"Set
mget task claim strategy as the default (#197070)\n\nResolves
https://github.com/elastic/kibana/issues/194625\r\n\r\nIn this PR, I'm
setting `mget` as the default task claiming strategy\r\nalong the
following changes:\r\n- Given we no longer need the 8.16 specific
PRs\r\n(#196317
and\r\nhttps://github.com//pull/196757), I've also
reverted them.\r\n- Given we now use `met` as the default, I've
renamed\r\n`task_manager_claimer_mget` to
`task_manager_claimer_update_by_query`\r\nand made tests in that folder
test using the `update_by_query` claim\r\nstrategy.\r\n- Stabilize flaky
tests caused by mget + polling for tasks more\r\nfrequently\r\n\r\nFlaky
test
runners:\r\n-\r\n[[59b71bc](https://github.com/elastic/kibana/pull/197070/commits/59b71bcdbe4d617a6d91131976540b334c9220ff)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7197\r\n-\r\n[[aea910e](https://github.com/elastic/kibana/pull/197070/commits/aea910e36dc71116dee708a7168971df30a18a3d)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7199\r\n-\r\n[[4723ced](https://github.com/elastic/kibana/pull/197070/commits/4723ced751f0e5114a9bc7a2928dcf0cb326472e)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7206\r\n-\r\n[[d28c8c5](https://github.com/elastic/kibana/pull/197070/commits/d28c8c56f67802107c17a627357251b9eff797ba)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7209\r\n-\r\n[[dd7773a](https://github.com/elastic/kibana/pull/197070/commits/dd7773aebad5664e725c9849c0ed9418f9dc68ed)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7224\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"c31f11e7d8c5c586258399c5e702e2247e05d0e4","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Feature:Task
Manager","Team:ResponseOps","v9.0.0","backport:prev-minor","v8.17.0"],"title":"Set
mget task claim strategy as the
default","number":197070,"url":"https://github.com/elastic/kibana/pull/197070","mergeCommit":{"message":"Set
mget task claim strategy as the default (#197070)\n\nResolves
https://github.com/elastic/kibana/issues/194625\r\n\r\nIn this PR, I'm
setting `mget` as the default task claiming strategy\r\nalong the
following changes:\r\n- Given we no longer need the 8.16 specific
PRs\r\n(#196317
and\r\nhttps://github.com//pull/196757), I've also
reverted them.\r\n- Given we now use `met` as the default, I've
renamed\r\n`task_manager_claimer_mget` to
`task_manager_claimer_update_by_query`\r\nand made tests in that folder
test using the `update_by_query` claim\r\nstrategy.\r\n- Stabilize flaky
tests caused by mget + polling for tasks more\r\nfrequently\r\n\r\nFlaky
test
runners:\r\n-\r\n[[59b71bc](https://github.com/elastic/kibana/pull/197070/commits/59b71bcdbe4d617a6d91131976540b334c9220ff)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7197\r\n-\r\n[[aea910e](https://github.com/elastic/kibana/pull/197070/commits/aea910e36dc71116dee708a7168971df30a18a3d)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7199\r\n-\r\n[[4723ced](https://github.com/elastic/kibana/pull/197070/commits/4723ced751f0e5114a9bc7a2928dcf0cb326472e)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7206\r\n-\r\n[[d28c8c5](https://github.com/elastic/kibana/pull/197070/commits/d28c8c56f67802107c17a627357251b9eff797ba)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7209\r\n-\r\n[[dd7773a](https://github.com/elastic/kibana/pull/197070/commits/dd7773aebad5664e725c9849c0ed9418f9dc68ed)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7224\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"c31f11e7d8c5c586258399c5e702e2247e05d0e4"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/197070","number":197070,"mergeCommit":{"message":"Set
mget task claim strategy as the default (#197070)\n\nResolves
https://github.com/elastic/kibana/issues/194625\r\n\r\nIn this PR, I'm
setting `mget` as the default task claiming strategy\r\nalong the
following changes:\r\n- Given we no longer need the 8.16 specific
PRs\r\n(#196317
and\r\nhttps://github.com//pull/196757), I've also
reverted them.\r\n- Given we now use `met` as the default, I've
renamed\r\n`task_manager_claimer_mget` to
`task_manager_claimer_update_by_query`\r\nand made tests in that folder
test using the `update_by_query` claim\r\nstrategy.\r\n- Stabilize flaky
tests caused by mget + polling for tasks more\r\nfrequently\r\n\r\nFlaky
test
runners:\r\n-\r\n[[59b71bc](https://github.com/elastic/kibana/pull/197070/commits/59b71bcdbe4d617a6d91131976540b334c9220ff)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7197\r\n-\r\n[[aea910e](https://github.com/elastic/kibana/pull/197070/commits/aea910e36dc71116dee708a7168971df30a18a3d)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7199\r\n-\r\n[[4723ced](https://github.com/elastic/kibana/pull/197070/commits/4723ced751f0e5114a9bc7a2928dcf0cb326472e)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7206\r\n-\r\n[[d28c8c5](https://github.com/elastic/kibana/pull/197070/commits/d28c8c56f67802107c17a627357251b9eff797ba)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7209\r\n-\r\n[[dd7773a](https://github.com/elastic/kibana/pull/197070/commits/dd7773aebad5664e725c9849c0ed9418f9dc68ed)]\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7224\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"c31f11e7d8c5c586258399c5e702e2247e05d0e4"}},{"branch":"8.x","label":"v8.17.0","branchLabelMappingKey":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Mike Côté <mikecote@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:prev-minor Backport to (9.0) the previous minor version (i.e. one version back from main) Feature:Task Manager release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v8.17.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make mget task claim strategy the default
7 participants