Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky test homepage app sample data uninstall should uninstall sample data set #20738

Closed
LeeDr opened this issue Jul 12, 2018 · 1 comment · Fixed by #21655
Closed

Flaky test homepage app sample data uninstall should uninstall sample data set #20738

LeeDr opened this issue Jul 12, 2018 · 1 comment · Fixed by #21655
Assignees
Labels
bug Fixes for quality problems that affect the customer experience failed-test A test failure on a tracked branch, potentially flaky-test Feature:Add Data Add Data and sample data feature on Home PR sent Team:Visualizations Visualization editors, elastic-charts and infrastructure test

Comments

@LeeDr
Copy link
Contributor

LeeDr commented Jul 12, 2018

Kibana version: master

Elasticsearch version: 7.0.0-alpha1

Server OS version: Ubuntu 16.04 (Jenkins worker)

Browser version: Chrome

Browser OS version:

Original install method (e.g. download page, yum, from source, etc.):

Describe the bug:
This test failed on Jenkins with Error: expected true to equal false which is from the expect right after this line;
https://github.com/elastic/kibana/blob/master/test/functional/apps/home/_sample_data.js#L106

We can see from the failure screenshot that the notification "Sample flight data uninstalled" is visible, but it also still shows "Sample flight data" is installed. Is it just a timing problem and has to keep checking for a bit to see if it changes state?
The previous expect in this test is within a retry.try() loop, and maybe this check should also be?

Steps to reproduce:

  1. run a PR with only a Docs change (nothing that should impact this test)
  2. this test failed :-(

I don't know how often this test fails?

Expected behavior:
This test should pass unless someone broke the code

Screenshots (if relevant):
homepage app sample data uninstall should uninstall sample data set

homepage app sample data uninstall should uninstall sample data set.html.txt

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):

16:32:33    │ proc  [ftr]      └-: uninstall
16:32:33    │ proc  [ftr]        └-> "before all" hook
16:32:33    │ proc  [ftr]        └-> should uninstall sample data set
16:32:33    │ proc  [ftr]          └-> "before each" hook: global before each
16:32:33    │ proc  [ftr]          │ debg  TestSubjects.click(removeSampleDataSetflights)
16:32:33    │ proc  [ftr]          │ debg  TestSubjects.find(removeSampleDataSetflights)
16:32:33    │ proc  [ftr]          │ debg  findByCssSelector [data-test-subj~="removeSampleDataSetflights"]
16:32:33    │ info  [o.e.c.m.MetaDataDeleteIndexService] [VuWBRUp] [kibana_sample_data_flights/_N7zXZFJSS2WNceKuMcYnw] deleting index
16:32:33    │ proc  [ftr]          │ debg  TestSubjects.exists(sampleDataSetUninstallToast)
16:32:33    │ proc  [ftr]          │ debg  existsByDisplayedByCssSelector [data-test-subj~="sampleDataSetUninstallToast"]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:33    │ info  [o.e.c.m.MetaDataIndexTemplateService] [VuWBRUp] adding template [kibana_index_template:.kibana] for index patterns [.kibana]
16:32:34    │ proc  [ftr]          │ debg  TestSubjects.exists(removeSampleDataSetflights)
16:32:34    │ proc  [ftr]          │ debg  existsByDisplayedByCssSelector [data-test-subj~="removeSampleDataSetflights"]
16:32:34    │ proc  [ftr]          │ info  Taking screenshot "/var/lib/jenkins/workspace/elastic+kibana+pull-request+multijob-selenium/kibana/test/functional/screenshots/failure/homepage app sample data uninstall should uninstall sample data set.png"
16:32:34    │ proc  [ftr]          │ info  Current URL is: http://localhost:5620/app/kibana#/home/tutorial_directory/sampleData?_g=()
16:32:34    │ proc  [ftr]          │ info  Saving page source to: /var/lib/jenkins/workspace/elastic+kibana+pull-request+multijob-selenium/kibana/test/functional/failure_debug/html/homepage app sample data uninstall should uninstall sample data set.html
16:32:34    │ proc  [ftr]        └- ✖ fail: "homepage app sample data uninstall should uninstall sample data set"
16:32:34    │ proc  [ftr]        │        Error: expected true to equal false
16:32:34    │ proc  [ftr]        │         at Assertion.assert (node_modules/expect.js/index.js:96:13)
16:32:34    │ proc  [ftr]        │         at Assertion.be.Assertion.equal (node_modules/expect.js/index.js:216:10)
16:32:34    │ proc  [ftr]        │         at Assertion.(anonymous function) [as be] (node_modules/expect.js/index.js:69:24)
16:32:34    │ proc  [ftr]        │         at Context.it (test/functional/apps/home/_sample_data.js:107:32)
16:32:34    │ proc  [ftr]        │         at <anonymous>
16:32:34    │ proc  [ftr]        │         at process._tickCallback (internal/process/next_tick.js:188:7)
16:32:34    │ proc  [ftr]        │       
16:32:34    │ proc  [ftr]        │       
16:32:34    │ proc  [ftr]        └-> "after all" hook
16:32:34    │ proc  [ftr]      └-> "after all" hook
16:32:34    │ proc  [ftr]    └-> "after all" hook

Any additional context:

@LeeDr LeeDr added bug Fixes for quality problems that affect the customer experience test :Management Feature:Add Data Add Data and sample data feature on Home and removed :Management labels Jul 12, 2018
@LeeDr LeeDr changed the title Flakey test homepage app sample data uninstall should uninstall sample data set Flaky test homepage app sample data uninstall should uninstall sample data set Jul 12, 2018
@nreese nreese self-assigned this Jul 25, 2018
@liza-mae
Copy link
Contributor

liza-mae commented Aug 3, 2018

Also seeing this fail on 6.4.0 BC2 on cloud.

Link to console:
https://internal-ci.elastic.co/view/Stack%20Tests/job/elastic+estf-cloud-kibana+master/4/consoleFull

@nreese nreese added the PR sent label Aug 3, 2018
@timroes timroes added Team:Visualizations Visualization editors, elastic-charts and infrastructure and removed :Sharing labels Sep 13, 2018
@liza-mae liza-mae added failed-test A test failure on a tracked branch, potentially flaky-test and removed flaky-failing-test labels Nov 1, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience failed-test A test failure on a tracked branch, potentially flaky-test Feature:Add Data Add Data and sample data feature on Home PR sent Team:Visualizations Visualization editors, elastic-charts and infrastructure test
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants