[EDR Workflows] Unskip and fix flaky endpoint exceptions FTR #197457
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gergoabraham
added
release_note:skip
This reverts commit b5b1052.
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#7227[✅] x-pack/test/security_solution_endpoint/configs/integrations.config.ts: 50/50 tests passed. |
|
Pinging @elastic/security-defend-workflows (Team:Defend Workflows) |
gergoabraham
requested review from
ashokaditya
and removed request for
parkiino
|
@elasticmachine merge upstream |
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]
History
|
|
Starting backport for target branches: 8.15, 8.16, 8.x https://github.com/elastic/kibana/actions/runs/11501404084 |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Oct 24, 2024
…#197457) ## Summary closes elastic#173184 closes elastic#173441 closes elastic#196003 This PR tries to improve on the `StaleElementReferenceError` happening in Endpoint Exception tests. This error is thrown if an element has already been removed from the DOM when trying to perform an action on it. For some reference, see elastic#140427 Improvements: - the part that was failing is wrapped inside the `retryOnStale` helper: 602f229 **note:** actually the test fails have started in December, 2023, but the line where the fail was in the last test runs were added in May, 2024 (elastic#183471). unfortunately, the log artifacts from 2023 are already removed from Buildkite, so no certainty on what happened back then - another suspicious part was wrapped as well: ec8c5cf and e5245ad - and as an extra, wait for page load: 7cd867f flaky 50/50 ✅ but this doesn't mean much, as this issue happens quite rarely ¯\\(◉‿◉)/¯ ### Checklist Delete any items that are not applicable to this PR. - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> (cherry picked from commit f151e2c)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Oct 24, 2024
…#197457) ## Summary closes elastic#173184 closes elastic#173441 closes elastic#196003 This PR tries to improve on the `StaleElementReferenceError` happening in Endpoint Exception tests. This error is thrown if an element has already been removed from the DOM when trying to perform an action on it. For some reference, see elastic#140427 Improvements: - the part that was failing is wrapped inside the `retryOnStale` helper: 602f229 **note:** actually the test fails have started in December, 2023, but the line where the fail was in the last test runs were added in May, 2024 (elastic#183471). unfortunately, the log artifacts from 2023 are already removed from Buildkite, so no certainty on what happened back then - another suspicious part was wrapped as well: ec8c5cf and e5245ad - and as an extra, wait for page load: 7cd867f flaky 50/50 ✅ but this doesn't mean much, as this issue happens quite rarely ¯\\(◉‿◉)/¯ ### Checklist Delete any items that are not applicable to this PR. - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> (cherry picked from commit f151e2c)
💔 Some backports could not be created
Note: Successful backport PRs will be merged automatically after passing CI. Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Oct 24, 2024
…197457) (#197662) # Backport This will backport the following commits from `main` to `8.x`: - [[EDR Workflows] Unskip and fix flaky endpoint exceptions FTR (#197457)](#197457) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Gergő Ábrahám","email":"gergo.abraham@elastic.co"},"sourceCommit":{"committedDate":"2024-10-24T14:27:08Z","message":"[EDR Workflows] Unskip and fix flaky endpoint exceptions FTR (#197457)\n\n## Summary\r\n\r\ncloses #173184 \r\ncloses #173441\r\ncloses #196003\r\n\r\nThis PR tries to improve on the `StaleElementReferenceError` happening\r\nin Endpoint Exception tests.\r\n\r\nThis error is thrown if an element has already been removed from the DOM\r\nwhen trying to perform an action on it. For some reference, see\r\nhttps://github.com//pull/140427\r\n\r\nImprovements:\r\n- the part that was failing is wrapped inside the `retryOnStale` helper:\r\n602f2294fddb9bee8b69ebf2fd8382e9f025d59d\r\n**note:** actually the test fails have started in December, 2023, but\r\nthe line where the fail was in the last test runs were added in May,\r\n2024 (#183471). unfortunately, the\r\nlog artifacts from 2023 are already removed from Buildkite, so no\r\ncertainty on what happened back then\r\n- another suspicious part was wrapped as well:\r\nec8c5cfd94812c8e5b357e00aac8bfae93ceecf4 and\r\ne5245ad010a02527105a56973465a25feb52ec85\r\n- and as an extra, wait for page load:\r\n7cd867fcb9489b24e79066dce750a2381af93d7d\r\n\r\nflaky 50/50 ✅ but this doesn't mean much, as this issue happens quite\r\nrarely ¯\\\\(◉‿◉)/¯\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"f151e2ccaa55cc5e13740f49e88c323c0e1d8f6d","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Defend Workflows","backport:prev-major"],"title":"[EDR Workflows] Unskip and fix flaky endpoint exceptions FTR","number":197457,"url":"https://github.com/elastic/kibana/pull/197457","mergeCommit":{"message":"[EDR Workflows] Unskip and fix flaky endpoint exceptions FTR (#197457)\n\n## Summary\r\n\r\ncloses #173184 \r\ncloses #173441\r\ncloses #196003\r\n\r\nThis PR tries to improve on the `StaleElementReferenceError` happening\r\nin Endpoint Exception tests.\r\n\r\nThis error is thrown if an element has already been removed from the DOM\r\nwhen trying to perform an action on it. For some reference, see\r\nhttps://github.com//pull/140427\r\n\r\nImprovements:\r\n- the part that was failing is wrapped inside the `retryOnStale` helper:\r\n602f2294fddb9bee8b69ebf2fd8382e9f025d59d\r\n**note:** actually the test fails have started in December, 2023, but\r\nthe line where the fail was in the last test runs were added in May,\r\n2024 (#183471). unfortunately, the\r\nlog artifacts from 2023 are already removed from Buildkite, so no\r\ncertainty on what happened back then\r\n- another suspicious part was wrapped as well:\r\nec8c5cfd94812c8e5b357e00aac8bfae93ceecf4 and\r\ne5245ad010a02527105a56973465a25feb52ec85\r\n- and as an extra, wait for page load:\r\n7cd867fcb9489b24e79066dce750a2381af93d7d\r\n\r\nflaky 50/50 ✅ but this doesn't mean much, as this issue happens quite\r\nrarely ¯\\\\(◉‿◉)/¯\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"f151e2ccaa55cc5e13740f49e88c323c0e1d8f6d"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/197457","number":197457,"mergeCommit":{"message":"[EDR Workflows] Unskip and fix flaky endpoint exceptions FTR (#197457)\n\n## Summary\r\n\r\ncloses #173184 \r\ncloses #173441\r\ncloses #196003\r\n\r\nThis PR tries to improve on the `StaleElementReferenceError` happening\r\nin Endpoint Exception tests.\r\n\r\nThis error is thrown if an element has already been removed from the DOM\r\nwhen trying to perform an action on it. For some reference, see\r\nhttps://github.com//pull/140427\r\n\r\nImprovements:\r\n- the part that was failing is wrapped inside the `retryOnStale` helper:\r\n602f2294fddb9bee8b69ebf2fd8382e9f025d59d\r\n**note:** actually the test fails have started in December, 2023, but\r\nthe line where the fail was in the last test runs were added in May,\r\n2024 (#183471). unfortunately, the\r\nlog artifacts from 2023 are already removed from Buildkite, so no\r\ncertainty on what happened back then\r\n- another suspicious part was wrapped as well:\r\nec8c5cfd94812c8e5b357e00aac8bfae93ceecf4 and\r\ne5245ad010a02527105a56973465a25feb52ec85\r\n- and as an extra, wait for page load:\r\n7cd867fcb9489b24e79066dce750a2381af93d7d\r\n\r\nflaky 50/50 ✅ but this doesn't mean much, as this issue happens quite\r\nrarely ¯\\\\(◉‿◉)/¯\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"f151e2ccaa55cc5e13740f49e88c323c0e1d8f6d"}}]}] BACKPORT--> Co-authored-by: Gergő Ábrahám <gergo.abraham@elastic.co>
kibanamachine
added a commit
that referenced
this pull request
Oct 24, 2024
…197457) (#197661) # Backport This will backport the following commits from `main` to `8.16`: - [[EDR Workflows] Unskip and fix flaky endpoint exceptions FTR (#197457)](#197457) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Gergő Ábrahám","email":"gergo.abraham@elastic.co"},"sourceCommit":{"committedDate":"2024-10-24T14:27:08Z","message":"[EDR Workflows] Unskip and fix flaky endpoint exceptions FTR (#197457)\n\n## Summary\r\n\r\ncloses #173184 \r\ncloses #173441\r\ncloses #196003\r\n\r\nThis PR tries to improve on the `StaleElementReferenceError` happening\r\nin Endpoint Exception tests.\r\n\r\nThis error is thrown if an element has already been removed from the DOM\r\nwhen trying to perform an action on it. For some reference, see\r\nhttps://github.com//pull/140427\r\n\r\nImprovements:\r\n- the part that was failing is wrapped inside the `retryOnStale` helper:\r\n602f2294fddb9bee8b69ebf2fd8382e9f025d59d\r\n**note:** actually the test fails have started in December, 2023, but\r\nthe line where the fail was in the last test runs were added in May,\r\n2024 (#183471). unfortunately, the\r\nlog artifacts from 2023 are already removed from Buildkite, so no\r\ncertainty on what happened back then\r\n- another suspicious part was wrapped as well:\r\nec8c5cfd94812c8e5b357e00aac8bfae93ceecf4 and\r\ne5245ad010a02527105a56973465a25feb52ec85\r\n- and as an extra, wait for page load:\r\n7cd867fcb9489b24e79066dce750a2381af93d7d\r\n\r\nflaky 50/50 ✅ but this doesn't mean much, as this issue happens quite\r\nrarely ¯\\\\(◉‿◉)/¯\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"f151e2ccaa55cc5e13740f49e88c323c0e1d8f6d","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Defend Workflows","backport:prev-major"],"title":"[EDR Workflows] Unskip and fix flaky endpoint exceptions FTR","number":197457,"url":"https://github.com/elastic/kibana/pull/197457","mergeCommit":{"message":"[EDR Workflows] Unskip and fix flaky endpoint exceptions FTR (#197457)\n\n## Summary\r\n\r\ncloses #173184 \r\ncloses #173441\r\ncloses #196003\r\n\r\nThis PR tries to improve on the `StaleElementReferenceError` happening\r\nin Endpoint Exception tests.\r\n\r\nThis error is thrown if an element has already been removed from the DOM\r\nwhen trying to perform an action on it. For some reference, see\r\nhttps://github.com//pull/140427\r\n\r\nImprovements:\r\n- the part that was failing is wrapped inside the `retryOnStale` helper:\r\n602f2294fddb9bee8b69ebf2fd8382e9f025d59d\r\n**note:** actually the test fails have started in December, 2023, but\r\nthe line where the fail was in the last test runs were added in May,\r\n2024 (#183471). unfortunately, the\r\nlog artifacts from 2023 are already removed from Buildkite, so no\r\ncertainty on what happened back then\r\n- another suspicious part was wrapped as well:\r\nec8c5cfd94812c8e5b357e00aac8bfae93ceecf4 and\r\ne5245ad010a02527105a56973465a25feb52ec85\r\n- and as an extra, wait for page load:\r\n7cd867fcb9489b24e79066dce750a2381af93d7d\r\n\r\nflaky 50/50 ✅ but this doesn't mean much, as this issue happens quite\r\nrarely ¯\\\\(◉‿◉)/¯\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"f151e2ccaa55cc5e13740f49e88c323c0e1d8f6d"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/197457","number":197457,"mergeCommit":{"message":"[EDR Workflows] Unskip and fix flaky endpoint exceptions FTR (#197457)\n\n## Summary\r\n\r\ncloses #173184 \r\ncloses #173441\r\ncloses #196003\r\n\r\nThis PR tries to improve on the `StaleElementReferenceError` happening\r\nin Endpoint Exception tests.\r\n\r\nThis error is thrown if an element has already been removed from the DOM\r\nwhen trying to perform an action on it. For some reference, see\r\nhttps://github.com//pull/140427\r\n\r\nImprovements:\r\n- the part that was failing is wrapped inside the `retryOnStale` helper:\r\n602f2294fddb9bee8b69ebf2fd8382e9f025d59d\r\n**note:** actually the test fails have started in December, 2023, but\r\nthe line where the fail was in the last test runs were added in May,\r\n2024 (#183471). unfortunately, the\r\nlog artifacts from 2023 are already removed from Buildkite, so no\r\ncertainty on what happened back then\r\n- another suspicious part was wrapped as well:\r\nec8c5cfd94812c8e5b357e00aac8bfae93ceecf4 and\r\ne5245ad010a02527105a56973465a25feb52ec85\r\n- and as an extra, wait for page load:\r\n7cd867fcb9489b24e79066dce750a2381af93d7d\r\n\r\nflaky 50/50 ✅ but this doesn't mean much, as this issue happens quite\r\nrarely ¯\\\\(◉‿◉)/¯\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"f151e2ccaa55cc5e13740f49e88c323c0e1d8f6d"}}]}] BACKPORT--> Co-authored-by: Gergő Ábrahám <gergo.abraham@elastic.co>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
closes #173184
closes #173441
closes #196003
This PR tries to improve on the
StaleElementReferenceErrorhappening in Endpoint Exception tests.This error is thrown if an element has already been removed from the DOM when trying to perform an action on it. For some reference, see #140427
Improvements:
retryOnStalehelper: 602f229note: actually the test fails have started in December, 2023, but the line where the fail was in the last test runs were added in May, 2024 ([EDR Workflows][tests] Enable endpoint exceptions integration test #183471). unfortunately, the log artifacts from 2023 are already removed from Buildkite, so no certainty on what happened back then
flaky 50/50 ✅ but this doesn't mean much, as this issue happens quite rarely ¯\(◉‿◉)/¯
Checklist
Delete any items that are not applicable to this PR.