Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/dashboard/_async_dashboard·ts - dashboard sample data dashboard "after all" hook for "toggle from Discover to Dashboard attempt 2"

[kibanamachine]

A test failed on a tracked branch

Error: expected true to equal false
    at Assertion.assert (/dev/shm/workspace/kibana/packages/kbn-expect/expect.js:100:11)
    at Assertion.be.Assertion.equal (/dev/shm/workspace/kibana/packages/kbn-expect/expect.js:227:8)
    at Assertion.be (/dev/shm/workspace/kibana/packages/kbn-expect/expect.js:69:22)
    at Context.after (test/functional/apps/dashboard/_async_dashboard.ts:109:30)

First failure: Jenkins Build

[kibanamachine]

New failure: Jenkins Build

[elasticmachine]

Pinging @elastic/kibana-test-triage (failed-test)

[kibanamachine]

New failure: Jenkins Build

[kibanamachine]

New failure: Jenkins Build

[kibanamachine]

New failure: Jenkins Build

[kibanamachine]

New failure: Jenkins Build

[kibanamachine]

New failure: Jenkins Build

[kibanamachine]

New failure: Jenkins Build

[kibanamachine]

New failure: Jenkins Build

[Dosant]

should be fixed by: #65976

[kibanamachine]

New failure: Jenkins Build

[Dosant]

failure above is due to #65976 is not backported yet

[kibanamachine]

New failure: Jenkins Build

[Dosant]

running on 7.x: https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/448/

[kibanamachine]

New failure: Jenkins Build

[Dosant]

1 failure out of 50 runs ⬆️ https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/448/ on 7.x

@LeeDr, @dmlemeshko, any ideas what else could cause flakiness? 🤔
Looks like there is still a bit of flakiness in after phase even after this stabilization which reduced flakiness #65976 significantly
Interesting that it seems to be flaky only on 7.x; master / 7.8 are somehow fine. - happened in master also https://kibana-ci.elastic.co/job/elastic+kibana+master/5101/

failure happens in after phase when cleaning up sample_flights set:

after(async () => {
      await PageObjects.common.navigateToUrl('home', 'tutorial_directory/sampleData');
      await PageObjects.home.removeSampleDataSet('flights');
      const isInstalled = await PageObjects.home.isSampleDataSetInstalled('flights');
      expect(isInstalled).to.be(false); // fails here !
    });

[kibanamachine]

New failure: Jenkins Build

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[LeeDr]

From the last test failure log, there's no indication that it failed to find and click the remove link. And yet the screenshot appears that it has not been clicked;

[00:10:01]               │ debg TestSubjects.click(removeSampleDataSetflights)
[00:10:01]               │ debg Find.clickByCssSelector('[data-test-subj="removeSampleDataSetflights"]') with timeout=10000
[00:10:01]               │ debg Find.findByCssSelector('[data-test-subj="removeSampleDataSetflights"]') with timeout=10000
[00:10:01]               │ debg TestSubjects.find(sampleDataSetCardflights)
[00:10:01]               │ debg Find.findByCssSelector('[data-test-subj="sampleDataSetCardflights"]') with timeout=10000
[00:10:03]               │ debg TestSubjects.exists(addSampleDataSetflights)
[00:10:03]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="addSampleDataSetflights"]') with timeout=2500
[00:10:05]               │ debg --- retry.tryForTime error: [data-test-subj="addSampleDataSetflights"] is not displayed
[00:10:06]               │ info Taking screenshot "/dev/shm/workspace/kibana/x-pack/test/functional/screenshots/failure/dashboard sample data dashboard _after all_ hook.png"
[00:10:06]               │ info Current URL is: http://localhost:6181/app/kibana#/home/tutorial_directory/sampleData
[00:10:06]               │ info Saving page source to: /dev/shm/workspace/kibana/x-pack/test/functional/failure_debug/html/dashboard sample data dashboard _after all_ hook.html
[00:10:06]               └- ✖ fail: "dashboard sample data dashboard "after all" hook for "toggle from Discover to Dashboard attempt 2""

When you click Remove, there's a short time when the screen changes to this;

And we don't see anything like that. It's like the click just didn't work.

[LeeDr]

I have a change to home_page here that might fix the flakiness; #66322

[kibanamachine]

New failure: Jenkins Build

[spalger]

This started failing a bunch of PRs very recently

Skipped

master: d9eae98
7.x/7.9: 49be10c
7.8: 0595789

[kibanamachine]

New failure: Jenkins Build

[kibanamachine]

New failure: Jenkins Build

[LeeDr]

In the home_page, we have these methods associated with removing sample data;

    async removeSampleDataSet(id: string) {
      // looks like overkill but we're hitting flaky cases where we click but it doesn't remove
      await testSubjects.waitForEnabled(`removeSampleDataSet${id}`);
      await testSubjects.click(`removeSampleDataSet${id}`);
      await this._waitForSampleDataLoadingAction(id);
    }

    // loading action is either uninstall and install
    async _waitForSampleDataLoadingAction(id: string) {
      const sampleDataCard = await testSubjects.find(`sampleDataSetCard${id}`);
      await retry.try(async () => {
        // waitForDeletedByCssSelector needs to be inside retry because it will timeout at least once
        // before action is complete
        await sampleDataCard.waitForDeletedByCssSelector('.euiLoadingSpinner');
      });
    }

One problem I see is the waiting for .euiLoadingSpinner. I'm on 7.8.1 right now and I don't see the euiLoadingSpinner appear in the DOM.
Here's a screenshot while it's removing. I do see something like looks like the spinner, but I don't see it in the DOM;

But that isn't where the problem is. It's not an issue of waiting for it to finish removing the sample data. As we see in the failure screenshot, it's a problem that it never starts to remove the sample data. The remove button normally changes very quickly as soon as it's clicked and should never go back until the data is re-added.

I'm sure that when the sample data page loads it checks if the user has the privs to add/remove sample data. Maybe there's a case where that check is still taking place but the button is enabled?

[kibanamachine]

New failure: Jenkins Build

[kibanamachine]

New failure: Jenkins Build

[kibanamachine]

New failure: Jenkins Build

[mistic]

That starts failing across master.

Skipping to unblock the CI.

[elasticmachine]

Pinging @elastic/kibana-app-arch (Team:AppArch)