Make visualization.waitForRender more generic #21342

Finally failed where I wanted it to fail: UI Functional Tests.test/functional/apps/dashboard/_dashboard_snapshots·js.dashboard app using current data dashboard snapshots compare area chart snapshot

Now to uncomment the waitForRender and verify that it passes.

elasticmachine · 2018-07-31T19:47:22Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-07-31T20:25:26Z

Failed because in the filtering test a visualization is saved with a different name, which makes running the same test multiple times in a row fail.

elasticmachine · 2018-07-31T21:53:18Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-08-01T03:09:03Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-08-01T19:34:45Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-08-02T19:47:26Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-08-02T21:23:24Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-08-03T14:40:08Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-08-03T19:43:29Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-08-03T22:38:15Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-08-04T00:27:30Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-08-04T00:31:42Z

Hm, failed on:

26:13    │ proc  [ftr]          │ debg  TestSubjects.click(toggleYAxisAdvancedOptions-ValueAxis-1)
00:26:13    │ proc  [ftr]          │ debg  TestSubjects.find(toggleYAxisAdvancedOptions-ValueAxis-1)
00:26:13    │ proc  [ftr]          │ debg  findByCssSelector [data-test-subj~="toggleYAxisAdvancedOptions-ValueAxis-1"]
00:26:13    │ proc  [ftr]          │ debg  TestSubjects.find(yAxisFilterLabelsCheckbox-ValueAxis-1)
00:26:13    │ proc  [ftr]          │ debg  findByCssSelector [data-test-subj~="yAxisFilterLabelsCheckbox-ValueAxis-1"]
00:26:13    │ proc  [ftr]          │ debg  TestSubjects.click(visualizeEditorRenderButton)
00:26:13    │ proc  [ftr]          │ debg  TestSubjects.find(visualizeEditorRenderButton)
00:26:13    │ proc  [ftr]          │ debg  findByCssSelector [data-test-subj~="visualizeEditorRenderButton"]
00:26:13    │ proc  [ftr]          │ debg  isGlobalLoadingIndicatorVisible
00:26:13    │ proc  [ftr]          │ debg  TestSubjects.exists(globalLoadingIndicator)
00:26:13    │ proc  [ftr]          │ debg  existsByDisplayedByCssSelector [data-test-subj~="globalLoadingIndicator"]
00:26:14    │ proc  [ftr]          │ debg  awaitGlobalLoadingIndicatorHidden
00:26:14    │ proc  [ftr]          │ debg  TestSubjects.find(globalLoadingIndicator-hidden)
00:26:14    │ proc  [ftr]          │ debg  findByCssSelector [data-test-subj~="globalLoadingIndicator-hidden"]
00:26:14    │ proc  [ftr]          │ debg  Visualization.waitForRender for 2 elements
00:26:14    │ proc  [ftr]          │ debg  in findAllByCssSelector: [data-render-complete="true"]
00:26:14    │ proc  [ftr]          │ debg  in findAllByCssSelector: [data-loading]
00:26:24    │ proc  [ftr]          │ debg  in findAllByCssSelector: .y > g
00:26:25    │ proc  [ftr]          │ info  Taking screenshot "/var/lib/jenkins/workspace/elastic+kibana+pull-request+multijob-selenium/kibana/test/functional/screenshots/failure/visualize app area charts switch between Y axis scale types should show ticks on selecting log scale.png"
00:26:25    │ proc  [ftr]          │ info  Current URL is: http://localhost:5620/app/kibana#/visualize/create?type=area&indexPattern=logstash-*&_g=(refreshInterval:(pause:!t,value:0),time:(from:%272015-09-19T06:31:44.000Z%27,mode:absolute,to:%272015-09-23T18:31:44.000Z%27))&_a=(filters:!(),linked:!f,query:(language:lucene,query:%27%27),uiState:(),vis:(aggs:!((enabled:!t,id:%271%27,params:(),schema:metric,type:count),(enabled:!t,id:%272%27,params:(customInterval:%272h%27,extended_bounds:(),field:%27@timestamp%27,interval:auto,min_doc_count:1),schema:segment,type:date_histogram)),params:(addLegend:!t,addTimeMarker:!f,addTooltip:!t,categoryAxes:!((id:CategoryAxis-1,labels:(show:!t,truncate:100),position:bottom,scale:(type:linear),show:!t,style:(),title:(),type:category)),grid:(categoryLines:!f,style:(color:%23eee)),legendPosition:right,seriesParams:!((data:(id:%271%27,label:Count),drawLinesBetweenPoints:!t,interpolate:linear,mode:stacked,show:true,showCircles:!t,type:area,valueAxis:ValueAxis-1)),times:!(),type:area,valueAxes:!((id:ValueAxis-1,labels:(filter:!f,rotate:0,show:!t,truncate:100),name:LeftAxis-1,position:left,scale:(mode:normal,type:log),show:!t,style:(),title:(text:Count),type:value))),title:%27New%20Visualization%27,type:area))
00:26:25    │ proc  [ftr]          │ info  Saving page source to: /var/lib/jenkins/workspace/elastic+kibana+pull-request+multijob-selenium/kibana/test/functional/failure_debug/html/visualize app area charts switch between Y axis scale types should show ticks on selecting log scale.html
00:26:25    │ proc  [ftr]        └- ✖ fail: "visualize app area charts switch between Y axis scale types should show ticks on selecting log scale"
00:26:25    │ proc  [ftr]        │       
00:26:25    │ proc  [ftr]        │         Error: expected [ '2',
00:26:25    │ proc  [ftr]        │         '3',
00:26:25    │ proc  [ftr]        │         '4',
00:26:25    │ proc  [ftr]        │         '6',
00:26:25    │ proc  [ftr]        │         '8',
00:26:25    │ proc  [ftr]        │         '20',
00:26:25    │ proc  [ftr]        │         '30',
00:26:25    │ proc  [ftr]        │         '40',
00:26:25    │ proc  [ftr]        │         '60',
00:26:25    │ proc  [ftr]        │         '80',
00:26:25    │ proc  [ftr]        │         '200',
00:26:25    │ proc  [ftr]        │         '300',
00:26:25    │ proc  [ftr]        │         '400',
00:26:25    │ proc  [ftr]        │         '600',
00:26:25    │ proc  [ftr]        │         '800',
00:26:25    │ proc  [ftr]        │         '2,000',
00:26:25    │ proc  [ftr]        │         '3,000',
00:26:25    │ proc  [ftr]        │         '4,000',
00:26:25    │ proc  [ftr]        │         '6,000',
00:26:25    │ proc  [ftr]        │         '8,000' ] to sort of equal [ '2',
00:26:25    │ proc  [ftr]        │         '3',
00:26:25    │ proc  [ftr]        │         '5',
00:26:25    │ proc  [ftr]        │         '7',
00:26:25    │ proc  [ftr]        │         '10',
00:26:25    │ proc  [ftr]        │         '20',
00:26:25    │ proc  [ftr]        │         '30',
00:26:25    │ proc  [ftr]        │         '50',
00:26:25    │ proc  [ftr]        │         '70',
00:26:25    │ proc  [ftr]        │         '100',
00:26:25    │ proc  [ftr]        │         '200',
00:26:25    │ proc  [ftr]        │         '300',
00:26:25    │ proc  [ftr]        │         '500',
00:26:25    │ proc  [ftr]        │         '700',
00:26:25    │ proc  [ftr]        │         '1,000',
00:26:25    │ proc  [ftr]        │         '2,000',
00:26:25    │ proc  [ftr]        │         '3,000',
00:26:25    │ proc  [ftr]        │         '5,000',
00:26:25    │ proc  [ftr]        │         '7,000' ]
00:26:25    │ proc  [ftr]        │         + expected - actual
00:26:25    │ proc  [ftr]        │       
00:26:25    │ proc  [ftr]        │          [
00:26:25    │ proc  [ftr]        │            "2"
00:26:25    │ proc  [ftr]        │            "3"
00:26:25    │ proc  [ftr]        │         -  "4"
00:26:25    │ proc  [ftr]        │         -  "6"
00:26:25    │ proc  [ftr]        │         -  "8"
00:26:25    │ proc  [ftr]        │         +  "5"
00:26:25    │ proc  [ftr]        │         +  "7"
00:26:25    │ proc  [ftr]        │         +  "10"
00:26:25    │ proc  [ftr]        │            "20"
00:26:25    │ proc  [ftr]        │            "30"
00:26:25    │ proc  [ftr]        │         -  "40"
00:26:25    │ proc  [ftr]        │         -  "60"
00:26:25    │ proc  [ftr]        │         -  "80"
00:26:25    │ proc  [ftr]        │         +  "50"
00:26:25    │ proc  [ftr]        │         +  "70"
00:26:25    │ proc  [ftr]        │         +  "100"
00:26:25    │ proc  [ftr]        │            "200"
00:26:25    │ proc  [ftr]        │            "300"
00:26:25    │ proc  [ftr]        │         -  "400"
00:26:25    │ proc  [ftr]        │         -  "600"
00:26:25    │ proc  [ftr]        │         -  "800"
00:26:25    │ proc  [ftr]        │         +  "500"
00:26:25    │ proc  [ftr]        │         +  "700"
00:26:25    │ proc  [ftr]        │         +  "1,000"
00:26:25    │ proc  [ftr]        │            "2,000"
00:26:25    │ proc  [ftr]        │            "3,000"
00:26:25    │ proc  [ftr]        │         -  "4,000"
00:26:25    │ proc  [ftr]        │         -  "6,000"
00:26:25    │ proc  [ftr]        │         -  "8,000"
00:26:25    │ proc  [ftr]        │         +  "5,000"
00:26:25    │ proc  [ftr]        │         +  "7,000"
00:26:25    │ proc  [ftr]        │          ]
00:26:25    │ proc  [ftr]        │         
00:26:25    │ proc  [ftr]        │         at Assertion.assert (node_modules/expect.js/index.js:96:13)
00:26:25    │ proc  [ftr]        │         at Assertion.eql (node_modules/expect.js/index.js:230:10)
00:26:25    │ proc  [ftr]        │         at Context.it (test/functional/apps/visualize/_area_chart.js:175:27)
00:26:25    │ proc  [ftr]        │         at <anonymous>
00:26:25    │ proc  [ftr]        │         at process._tickCallback (internal/process/next_tick.js:188:7)
00:26:25    │ proc  [ftr]        │       
00:26:25    │ proc  [ftr]        │       
00:26:25    │ proc  [ftr]        └-> "after all" hook

Didn't change any real code logic so this shouldn't have introduced a bug... Will try locally.

Ah, this is flaky (#21642):

jenkins, test this

stacey-gammon · 2018-08-04T00:33:45Z

retest

elasticmachine · 2018-08-04T02:03:31Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-08-04T12:36:27Z

Hit #21642 again

jenkins, test this

…e class.

… verify that flakiness is gone

…d later.

This should be fixed but for now I'd like to continue testing the rendering test refactor. The issue is that this _dashboard_filtering test modifies saved objects. The "using current data" tests do not reset the data after each suite run to try to cut back on the time tests take. With all the work infra is doing we can probably remove this little attempt at a clean up. For now though, stay focused on the task...

elasticmachine · 2018-08-04T13:51:51Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-08-04T16:14:58Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-08-04T17:05:22Z

retest

elasticmachine · 2018-08-04T18:54:09Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-08-07T15:06:50Z

closing in favor of #21688

stacey-gammon added test :Sharing labels Jul 27, 2018

stacey-gammon mentioned this pull request Jul 27, 2018

[WIP] Experiment on top of Timroes vis rendering wait #21299

Closed

stacey-gammon force-pushed the 2018-07-27-test-render-cleanup branch from b3ca2c7 to 79c561f Compare July 30, 2018 12:30

stacey-gammon force-pushed the 2018-07-27-test-render-cleanup branch from 79c561f to 579c2dd Compare July 30, 2018 19:58

stacey-gammon force-pushed the 2018-07-27-test-render-cleanup branch from 2aec48c to bc00c7f Compare July 31, 2018 18:20

stacey-gammon force-pushed the 2018-07-27-test-render-cleanup branch from c88cae4 to fda9a4e Compare August 1, 2018 18:12

stacey-gammon force-pushed the 2018-07-27-test-render-cleanup branch from fda9a4e to 55de839 Compare August 2, 2018 18:29

stacey-gammon added 11 commits August 4, 2018 08:40

Make visualization.waitForRender more generic and put in an embeddabl…

786f011

…e class.

fix other assertion when url is on visualize page, not dashboard

59d8ef2

Turn waitForRenderComplete on. Also run filtering tests many times to…

e5c9ec1

… verify that flakiness is gone

Re-saving the same visualization with dashs, hides it from being foun…

8dcdce2

…d later.

remove more references to visualize service, default to 2.

40db3c8

Use a new snapshot that doesn't seem to have the mouse hovering over

416a859

clean up after test so i can run multiple times in a row

56c2482

fixes

0329fe2

data tables only have one tag i guess

976224d

second spot needed waitForRender(1)

a6cab91

stacey-gammon force-pushed the 2018-07-27-test-render-cleanup branch from b2bdc2e to a6cab91 Compare August 4, 2018 12:40

use embeddable not visualization service

4a789a3

stacey-gammon mentioned this pull request Aug 6, 2018

Make visualization.waitForRender more generic and put in an embeddable class #21688

Merged

stacey-gammon closed this Aug 7, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make visualization.waitForRender more generic #21342

Make visualization.waitForRender more generic #21342

stacey-gammon commented Jul 27, 2018

elasticmachine commented Jul 27, 2018

elasticmachine commented Jul 30, 2018

elasticmachine commented Jul 30, 2018

elasticmachine commented Jul 30, 2018

elasticmachine commented Jul 30, 2018

elasticmachine commented Jul 31, 2018

stacey-gammon commented Jul 31, 2018

elasticmachine commented Jul 31, 2018

stacey-gammon commented Jul 31, 2018

elasticmachine commented Jul 31, 2018

elasticmachine commented Aug 1, 2018

elasticmachine commented Aug 1, 2018

elasticmachine commented Aug 2, 2018

elasticmachine commented Aug 2, 2018

elasticmachine commented Aug 3, 2018

elasticmachine commented Aug 3, 2018

elasticmachine commented Aug 3, 2018

elasticmachine commented Aug 4, 2018

stacey-gammon commented Aug 4, 2018 •

edited

Loading

stacey-gammon commented Aug 4, 2018

elasticmachine commented Aug 4, 2018

stacey-gammon commented Aug 4, 2018

elasticmachine commented Aug 4, 2018

elasticmachine commented Aug 4, 2018

stacey-gammon commented Aug 4, 2018

elasticmachine commented Aug 4, 2018

stacey-gammon commented Aug 7, 2018

Make visualization.waitForRender more generic #21342

Make visualization.waitForRender more generic #21342

Conversation

stacey-gammon commented Jul 27, 2018

elasticmachine commented Jul 27, 2018

💔 Build Failed

elasticmachine commented Jul 30, 2018

💔 Build Failed

elasticmachine commented Jul 30, 2018

💔 Build Failed

elasticmachine commented Jul 30, 2018

💔 Build Failed

elasticmachine commented Jul 30, 2018

💔 Build Failed

elasticmachine commented Jul 31, 2018

💔 Build Failed

stacey-gammon commented Jul 31, 2018

elasticmachine commented Jul 31, 2018

💔 Build Failed

stacey-gammon commented Jul 31, 2018

elasticmachine commented Jul 31, 2018

💔 Build Failed

elasticmachine commented Aug 1, 2018

💔 Build Failed

elasticmachine commented Aug 1, 2018

💔 Build Failed

elasticmachine commented Aug 2, 2018

💔 Build Failed

elasticmachine commented Aug 2, 2018

💔 Build Failed

elasticmachine commented Aug 3, 2018

💔 Build Failed

elasticmachine commented Aug 3, 2018

💔 Build Failed

elasticmachine commented Aug 3, 2018

💔 Build Failed

elasticmachine commented Aug 4, 2018

💔 Build Failed

stacey-gammon commented Aug 4, 2018 • edited Loading

stacey-gammon commented Aug 4, 2018

elasticmachine commented Aug 4, 2018

💔 Build Failed

stacey-gammon commented Aug 4, 2018

elasticmachine commented Aug 4, 2018

💔 Build Failed

elasticmachine commented Aug 4, 2018

💚 Build Succeeded

stacey-gammon commented Aug 4, 2018

elasticmachine commented Aug 4, 2018

💚 Build Succeeded

stacey-gammon commented Aug 7, 2018

stacey-gammon commented Aug 4, 2018 •

edited

Loading