Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Functional test] Increase the timeout on opening a saved visualization #70952

Merged
merged 2 commits into from
Jul 8, 2020
Merged

[Functional test] Increase the timeout on opening a saved visualization #70952

merged 2 commits into from
Jul 8, 2020

Conversation

stratoula
Copy link
Contributor

@stratoula stratoula commented Jul 7, 2020 •
edited
Loading

Summary

Fixes #68850. Adds a retry on open saved visualization.

Checklist

@stratoula stratoula changed the title fixes the flakiness on hybrid visualization test [Functional test] Add retry on visualization click Jul 7, 2020
@stratoula stratoula changed the title [Functional test] Add retry on visualization click [Functional test] Add retry on open saved visualization Jul 7, 2020
@stratoula
Copy link
Contributor Author

stratoula commented Jul 7, 2020 •
edited
Loading

This fails on the cloud (7.8). I have created a flaky test runner (https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/564/) and it seems to work. I have replicated it only once to the cloud. It seems that it doesn't find the visualization on the list so I suggest to add a retry here. When I added the retry couldn't replicate it but I am not 100% sure that it will solve the problem. @dmlemeshko what do you think ?

@stratoula stratoula marked this pull request as ready for review July 7, 2020 15:56
@stratoula stratoula added release_note:skip Skip the PR/issue when compiling release notes v7.9.0 v8.0.0 labels Jul 7, 2020
@stratoula stratoula requested a review from dmlemeshko July 7, 2020 16:01
@dmlemeshko
Copy link
Member

dmlemeshko commented Jul 7, 2020 •
edited
Loading

This fails on the cloud (7.8). I have created a flaky test runner (https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/564/) and it seems to work. I have replicated it only once to the cloud. It seems that it doesn't find the visualization on the list so I suggest to add a retry here. When I added the retry couldn't replicate it but I am not 100% sure that it will solve the problem. @dmlemeshko what do you think ?

Since it is a single action without any verification I don't think retry makes much sense here, we can easily just increase timeout to 20 sec: await testSubjects.click(dataTestSubj, 20000);
And if viz landing page is not loaded within 20 sec, it is cloud env issue we should raise.

I looked for the issues with test-cloud tag and many of them are failing with timeout, so as a general solution we might wanna try is to update FTR schema for the cloud run https://github.com/elastic/kibana/blob/master/packages/kbn-test/src/functional_test_runner/lib/config/schema.ts#L112, maybe 2x first?
@liza-mae what do you think about it?

@stratoula
Copy link
Contributor Author

@dmlemeshko you are right. I made your suggested change but I agree, maybe a more general approach would solve more test problems

dmlemeshko
dmlemeshko approved these changes Jul 8, 2020
View reviewed changes
Copy link
Member

@dmlemeshko dmlemeshko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM

@stratoula stratoula changed the title [Functional test] Add retry on open saved visualization [Functional test] Increase the timeout on opening a saved visualization Jul 8, 2020
@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Build metrics

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@stratoula stratoula merged commit 0ea7f9f into elastic:master Jul 8, 2020
stratoula added a commit to stratoula/kibana that referenced this pull request Jul 8, 2020
@stratoula
[Functional test] Increase the timeout on opening a saved visualizati…
25b1360 
…on (elastic#70952)

* fixes the flakiness on hybrid visualization test

* increase timeout to 20 sec to find and click the hybrid visualization
@stratoula stratoula mentioned this pull request Jul 8, 2020
Merged
stratoula added a commit that referenced this pull request Jul 8, 2020
@stratoula @elasticmachine
[Functional test] Increase the timeout on opening a saved visualizati…
f4747c1 
…on (#70952) (#71055)

* fixes the flakiness on hybrid visualization test

* increase timeout to 20 sec to find and click the hybrid visualization

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
gmmorris added a commit to gmmorris/kibana that referenced this pull request Jul 9, 2020
@gmmorris
Merge branch 'master' into actions/feature-and-rbac
f33ac25 
* master:
  [RUM Dashboard] New rum services api to replace usage of get services API (elastic#70746)
  fix: remove only consecutive ticks in TSVB (elastic#70981)
  [Functional test] Increase the timeout on opening a saved visualization (elastic#70952)
  [ML] Transforms/DFA: Refactor list action buttons so modals won't unmount after button click. (elastic#70555)
  [Functional test] Add retry for dashboard save (elastic#70950)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dmlemeshko dmlemeshko dmlemeshko approved these changes

Assignees
No one assigned
Labels
release_note:skip Skip the PR/issue when compiling release notes v7.9.0 v8.0.0
Projects
None yet
Milestone
No milestone
3 participants
@stratoula @dmlemeshko @kibanamachine