Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Functional tests - stabilize and re-enable test suites #144784

Merged
merged 6 commits into from
Nov 10, 2022

Conversation

pheyos
Copy link
Member

@pheyos pheyos commented Nov 8, 2022

Summary

This PR stabilizes and re-enables a few ML functional test suites.

Details

Re-enable pages test suite

Test ran into a blank page with [SEVERE] http://localhost:5620/57029/bundles/kbn-ui-shared-deps-npm/kbn-ui-shared-deps-npm.dll.js 398:83112 TypeError: Reduce of empty array with no initial value. Seems to be resolved now.
Closes #142397

Stabilize and re-enable lens-to-ml test suites

There was an issue with test cleanups, which could bring up confusing failure messages in follow-up tests. The stability fix is to move the job deletion to an after method. The root cause of the issue seems to be resolved already.
Closes #144186
Closes #142762

Stabilize and re-enable index data visualizer test suite

With the introduction of random sampler in the data visualizer, the total document count was sometimes not the accurate value. The stability fix is to disable random sampling for this assertion by default.
Closes #143007

Stabilize and re-enable custom URL test suite

After navigating to Discover, the test sometimes failed to close the Discover side bar when clicking it too early after navigating to the page. The stability fix is to wait for global loading to finish before closing the side bar.
Closes #143933

@pheyos
Copy link
Member Author

pheyos commented Nov 8, 2022

Checking test stability in a flaky test runner job ... no ML failure in 50 runs ✔️
(just one unrelated infrastructure issue that passed on 2nd try).

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Unknown metric groups

ESLint disabled in files

id before after diff
osquery 1 2 +1

ESLint disabled line counts

id before after diff
enterpriseSearch 19 21 +2
fleet 59 65 +6
osquery 108 113 +5
securitySolution 440 446 +6
total +19

Total ESLint disabled count

id before after diff
enterpriseSearch 20 22 +2
fleet 67 73 +6
osquery 109 115 +6
securitySolution 517 523 +6
total +20

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@pheyos pheyos self-assigned this Nov 8, 2022
@pheyos pheyos requested review from qn895 and jgowdyelastic November 8, 2022 16:48
@pheyos pheyos added :ml test_ui_functional release_note:skip Skip the PR/issue when compiling release notes backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) v8.6.0 v8.5.1 labels Nov 8, 2022
@pheyos pheyos marked this pull request as ready for review November 8, 2022 16:49
@pheyos pheyos requested a review from a team as a code owner November 8, 2022 16:49
@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui (:ml)

@qn895
Copy link
Member

qn895 commented Nov 9, 2022

LGTM 🎉

@pheyos pheyos merged commit cf281d6 into elastic:main Nov 10, 2022
@pheyos pheyos deleted the reenable_tests branch November 10, 2022 09:35
@kibanamachine
Copy link
Contributor

💔 All backports failed

Status Branch Result
8.5 Backport failed because of merge conflicts

You might need to backport the following PRs to 8.5:
- [ML] Add functional tests for Index data visualizer's random sampler controls (#142278)

Manual backport

To create the backport manually run:

node scripts/backport --pr 144784

Questions ?

Please refer to the Backport tool documentation

@pheyos pheyos removed backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) v8.5.1 labels Nov 10, 2022
@kibanamachine kibanamachine added the backport:skip This commit does not require backporting label Nov 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting :ml release_note:skip Skip the PR/issue when compiling release notes test_ui_functional v8.6.0
Projects
None yet
5 participants