Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing test: Serverless Observability Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/search_examples/partial_results_example·ts - serverless examples UI search examples Partial results example "before all" hook for "should update a progress bar" #165763

Closed
Tracked by #166037
kibanamachine opened this issue Sep 5, 2023 · 12 comments · Fixed by #166410 or #167497
Closed
Tracked by #166037

Failing test: Serverless Observability Examples Functional Tests.x-pack/test_serverless/functional/test_suites/common/examples/search_examples/partial_results_example·ts - serverless examples UI search examples Partial results example "before all" hook for "should update a progress bar" #165763

kibanamachine opened this issue Sep 5, 2023 · 12 comments · Fixed by #166410 or #167497
Assignees
@davismcphee
Labels
failed-test A test failure on a tracked branch, potentially flaky-test impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. loe:small Small Level of Effort Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Sep 5, 2023 •
edited
Loading

A test failed on a tracked branch

Error: retry.tryForTime timeout: Error: retry.try timeout: TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="kibanaChrome"] nav:not(.ng-hide))
Wait timed out after 61194ms
    at /var/lib/buildkite-agent/builds/kb-n2-4-spot-f85d4346644156b3/elastic/kibana-serverless/kibana/node_modules/selenium-webdriver/lib/webdriver.js:929:17
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at onFailure (retry_for_success.ts:17:9)
    at retryForSuccess (retry_for_success.ts:59:13)
    at RetryService.try (retry.ts:31:12)
    at common_page.ts:265:21
    at runAttempt (retry_for_success.ts:29:15)
    at retryForSuccess (retry_for_success.ts:68:21)
    at RetryService.tryForTime (retry.ts:22:12)
    at CommonPageObject.navigateToApp (common_page.ts:264:5)
    at Context.<anonymous> (partial_results_example.ts:18:7)
    at Object.apply (wrap_function.js:73:16)
    at onFailure (retry_for_success.ts:17:9)
    at retryForSuccess (retry_for_success.ts:59:13)
    at RetryService.tryForTime (retry.ts:22:12)
    at CommonPageObject.navigateToApp (common_page.ts:264:5)
    at Context.<anonymous> (partial_results_example.ts:18:7)
    at Object.apply (wrap_function.js:73:16)

First failure: CI Build - main
@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Sep 5, 2023
@botelastic botelastic bot added the needs-team Issues missing a team label label Sep 5, 2023
@kibanamachine kibanamachine added the Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. label Sep 5, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Sep 5, 2023
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

mistic added a commit that referenced this issue Sep 6, 2023
@mistic
skip flaky suite (#165763)
79e11c2
@mistic
Copy link
Member

mistic commented Sep 6, 2023

Skipped.

main: 79e11c2

@davismcphee davismcphee added loe:small Small Level of Effort impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. and removed blocker labels Sep 8, 2023
@davismcphee davismcphee self-assigned this Sep 8, 2023
This was referenced Sep 8, 2023
Closed
Merged
Closed
Closed
davismcphee added a commit that referenced this issue Sep 20, 2023
@davismcphee
[Search] Unskip partial results Serverless tests (#166410)
8e800ee 
## Summary

This PR unskips the partial results Serverless tests. I ran 100x flaky
test runs against each project type to confirm they should no longer be
flaky.

Flaky test runs:
- x100 (`.only`):
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3114
- x100 Search:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3121
- x100 O11y:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3147
- x100 Security:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3156

Resolves #165563.
Resolves #165612.
Resolves #165763.

### Checklist

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
@dmlemeshko dmlemeshko mentioned this issue Sep 28, 2023
Merged
@davismcphee davismcphee added impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. and removed impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. labels Sep 29, 2023
dmlemeshko added a commit that referenced this issue Sep 29, 2023
@dmlemeshko
[ftr] unskip serverless functional tests (#167497)
97c7d7d 
I hope the change made in #166936 allow us to unskip these tests

closes #165763
closes #165386
closes #165414
closes #166448 
closes #165943 
closes #166461
closes #166551

Flaky-test-runner:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3242

Quick stability check of re-enabled tests:
-
x-pack/test_serverless/functional/test_suites/common/examples/search_examples/partial_results_example.ts
100/100
-
x-pack/test_serverless/functional/test_suites/common/examples/unified_field_list_examples/existing_fields.ts
100/100 failures
- x-pack/test_serverless/functional/test_suites/common/home_page.ts
100/100
-
x-pack/test_serverless/functional/test_suites/common/index_management/create_enrich_policy.ts
99/100
-
x-pack/test_serverless/functional/test_suites/observability/cases/configure.ts
49/50
-
x-pack/test_serverless/functional/test_suites/observability/observability_log_explorer/app.ts
49/50
-
x-pack/test_serverless/functional/test_suites/observability/observability_log_explorer/filter_controls.ts
49/50
-
x-pack/test_serverless/functional/test_suites/security/ftr/cases/configure.ts
49/50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@davismcphee davismcphee

Labels
failed-test A test failure on a tracked branch, potentially flaky-test impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. loe:small Small Level of Effort Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.
Projects
None yet
Milestone
No milestone
4 participants
@mistic @elasticmachine @davismcphee @kibanamachine