Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution][Detections] Implements indicator match rule cypress test #84323

Merged
merged 11 commits into from
Dec 3, 2020

Conversation

MadameSheema
Copy link
Member

@MadameSheema MadameSheema commented Nov 25, 2020

Summary

In this PR we are adding a new test in order to check that an Indicator Match rule can be created and is able to populate alerts.

For this test we have created two different archives:

  • threat_data
  • threat indicator

The threat_data archive contains an auditbeat type document that matches with threat-data-* index pattern.

The threat_index archive, contains two documents that matches with threat-index-* index pattern that contains the threats, that matches with the threat-data.

The test:

  • Creates an Indicator Match rule using the UI
  • Checks that the created rule has all the expected data
  • Checks that the rule generates the expected alert and has the expected values.

@MadameSheema MadameSheema self-assigned this Nov 25, 2020
@MadameSheema MadameSheema added v7.11.0 Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detections and Resp Security Detection Response Team release_note:skip Skip the PR/issue when compiling release notes v8.0.0 labels Nov 25, 2020
@MadameSheema MadameSheema marked this pull request as ready for review November 25, 2020 15:06
@MadameSheema MadameSheema requested review from a team as code owners November 25, 2020 15:06
@MadameSheema
Copy link
Member Author

@elasticmachine merge upstream

@FrankHassanabad
Copy link
Contributor

Just a suggestion but es_archiver does support non gzip of data.

Looking at:

x-pack/test/security_solution_cypress/es_archives/threat_data/data.json.gz

That looks like it only has a single record within it, so I would just not gzip it up and check it in as data.json and then for the mapping.json here and that data.json:

 ~/projects/kibana/x-pack/test/security_solution_cypress/es_archives/threat_data/mappings.json

I would trim the mappings down if possible to only match the parts of the data set you need to match. Technically everything will work out ok if we use ECS compatible mappings even if they don't have the full data set including the full mapping. Also nothing wrong with adding something to the mappings that don't exist in ECS as long as we don't use a conflicting field name from ECS.

For example, on backend e2e tests I am using very small data sets and small mappings like these recent ones:
https://github.com/elastic/kibana/tree/master/x-pack/test/functional/es_archives/rule_exceptions

The only "required field" is @timestamp and really because ECS is a constant changing/evolving standard it was just easier for me to conduct tests that way. For example, soon ECS is going to change a lot of mappings to be wildcard:
https://github.com/elastic/ecs/releases

But in my above tests it shouldn't matter since I only use a @timestamp and a made-up non conflicting field. I didn't gzip anything up as the record contents were very small and so was the mappings. From a maintenance perspective I have been noticing more people with newer e2e tests on the backend such as the alerting team doing these patterns of not zipping things and using small data sets so I think it's catching on as being easier.

Optional here, do what's best, just mentioning this and depends on what you're testing. I typically these days try to keep the least amount of mapping/data for functional tests since ECS is constantly evolving. For other tests or issues with backwards compatibility with ECS I try to do the same thing and just concentrate on types such as keyword/wildcard, etc... and not directly test something large or the actual fields of ECS if I don't have to.

Although I might need to do more of those later if we have bugs directly related to different ECS mapping types. However, even then I try to abstract it away into something that doesn't directly involve ECS mappings if I can (other than @timestamp needing to be there).

That also usually catches bugs if someone is relying a field to exist from ECS as an added bonus since all those fields should be optional other than @tiemstamp

"type": "index",
"value": {
"aliases": {
"playing": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you want this here where the alias is playing?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed that, thanks @FrankHassanabad :)

return alertCount > 0;
});
},
{ interval: 500, timeout: 12000 }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was there a reason for this to be here previously, i.e. timing reasons? Are other tests relying on this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The method is on the same file but displayed in a different order. I just sorted the methods alphabetically.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't see parameters for interval and timeout in the other method. Were the values here used at all?

@MadameSheema
Copy link
Member Author

@FrankHassanabad I think I have simplified and fix the data:

  • I have fixed the alias name of the threat_data mapping
  • I have simplified the threat_index data and mapping, using just one doc and deleting all the fields we don't use for the test
  • I have change the threat_index to use a non ECS field to check the rule is able to work with this type of fields.

Can you please review the PR again, lots of thanks :)

@MadameSheema
Copy link
Member Author

@elasticmachine merge upstream

const alertCount = parseInt(countText, 10) || 0;
return alertCount > 0;
});
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

For the backend e2e, I started just like this, but then later I had to add a specific number to wait for a specific number of alerts as even with things like refresh=wait_for within the code there does exist some time before the Nth alert shows up.

I had some flakiness where 1 signal would show up on the backend but I needed to wait for like 2 or 3. No changes requested, but if we see flaky we will probably have to add that integer here.

Copy link
Contributor

@FrankHassanabad FrankHassanabad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for the tests here for exceptions. Users use exceptions A LOT and we are fixing bugs/refactoring parts of it so this is going to be so great for us to ensure that the exceptions are all operating as expected.

@kibanamachine
Copy link
Contributor

💛 Build succeeded, but was flaky


Test Failures

Chrome X-Pack UI Functional Tests.x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/alert_create_flyout·ts.Actions and Triggers app create alert should show save confirmation before creating alert with no actions

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:00:00]         └-: Actions and Triggers app
[00:00:00]           └-> "before all" hook
[00:03:49]           └-: create alert
[00:03:49]             └-> "before all" hook
[00:03:49]             └-> "before all" hook
[00:03:49]               │ debg navigating to triggersActions url: http://localhost:61111/app/management/insightsAndAlerting/triggersActions
[00:03:49]               │ debg navigate to: http://localhost:61111/app/management/insightsAndAlerting/triggersActions
[00:03:50]               │ debg browser[INFO] http://localhost:61111/app/management/insightsAndAlerting/triggersActions?_t=1606934658425 341 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:03:50]               │
[00:03:50]               │ debg browser[INFO] http://localhost:61111/bootstrap.js 42:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:03:50]               │ debg ... sleep(700) start
[00:03:50]               │ debg ... sleep(700) end
[00:03:50]               │ debg returned from get, calling refresh
[00:03:51]               │ debg browser[INFO] http://localhost:61111/app/management/insightsAndAlerting/triggersActions?_t=1606934658425 341 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:03:51]               │
[00:03:51]               │ debg browser[INFO] http://localhost:61111/bootstrap.js 42:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:03:51]               │ debg currentUrl = http://localhost:61111/app/management/insightsAndAlerting/triggersActions
[00:03:51]               │          appUrl = http://localhost:61111/app/management/insightsAndAlerting/triggersActions
[00:03:51]               │ debg TestSubjects.find(kibanaChrome)
[00:03:51]               │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:03:53]               │ debg ... sleep(501) start
[00:03:53]               │ debg ... sleep(501) end
[00:03:53]               │ debg in navigateTo url = http://localhost:61111/app/management/insightsAndAlerting/triggersActions/alerts
[00:03:53]               │ debg --- retry.try error: URL changed, waiting for it to settle
[00:03:54]               │ debg ... sleep(501) start
[00:03:54]               │ debg ... sleep(501) end
[00:03:54]               │ debg in navigateTo url = http://localhost:61111/app/management/insightsAndAlerting/triggersActions/alerts
[00:03:54]               │ debg TestSubjects.exists(statusPageContainer)
[00:03:54]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="statusPageContainer"]') with timeout=2500
[00:03:57]               │ debg --- retry.tryForTime error: [data-test-subj="statusPageContainer"] is not displayed
[00:03:57]               │ debg TestSubjects.click(alertsTab)
[00:03:57]               │ debg Find.clickByCssSelector('[data-test-subj="alertsTab"]') with timeout=10000
[00:03:57]               │ debg Find.findByCssSelector('[data-test-subj="alertsTab"]') with timeout=10000
[00:03:57]             └-> should create an alert
[00:03:57]               └-> "before each" hook: global before each
[00:03:57]               │ debg Find.findByCssSelector('[data-test-subj="createAlertButton"],[data-test-subj="createFirstAlertButton"]') with timeout=10000
[00:03:58]               │ debg TestSubjects.setValue(alertNameInput, 0ee1f45f829543af80922e1c6ebdf48f)
[00:03:58]               │ debg TestSubjects.click(alertNameInput)
[00:03:58]               │ debg Find.clickByCssSelector('[data-test-subj="alertNameInput"]') with timeout=10000
[00:03:58]               │ debg Find.findByCssSelector('[data-test-subj="alertNameInput"]') with timeout=10000
[00:03:59]               │ debg TestSubjects.click(.index-threshold-SelectOption)
[00:03:59]               │ debg Find.clickByCssSelector('[data-test-subj=".index-threshold-SelectOption"]') with timeout=10000
[00:03:59]               │ debg Find.findByCssSelector('[data-test-subj=".index-threshold-SelectOption"]') with timeout=10000
[00:03:59]               │ debg TestSubjects.click(selectIndexExpression)
[00:03:59]               │ debg Find.clickByCssSelector('[data-test-subj="selectIndexExpression"]') with timeout=10000
[00:03:59]               │ debg Find.findByCssSelector('[data-test-subj="selectIndexExpression"]') with timeout=10000
[00:03:59]               │ debg Find.findByCssSelector('#indexSelectSearchBox') with timeout=10000
[00:04:00]               │ debg Find.findByCssSelector('.euiFilterSelectItem') with timeout=10000
[00:04:00]               │ debg TestSubjects.click(thresholdAlertTimeFieldSelect)
[00:04:00]               │ debg Find.clickByCssSelector('[data-test-subj="thresholdAlertTimeFieldSelect"]') with timeout=10000
[00:04:00]               │ debg Find.findByCssSelector('[data-test-subj="thresholdAlertTimeFieldSelect"]') with timeout=10000
[00:04:00]               │ debg Find.allByCssSelector('#thresholdTimeField option') with timeout=10000
[00:04:00]               │ debg TestSubjects.click(closePopover)
[00:04:00]               │ debg Find.clickByCssSelector('[data-test-subj="closePopover"]') with timeout=10000
[00:04:00]               │ debg Find.findByCssSelector('[data-test-subj="closePopover"]') with timeout=10000
[00:04:00]               │ debg TestSubjects.find(alertNameInput)
[00:04:00]               │ debg Find.findByCssSelector('[data-test-subj="alertNameInput"]') with timeout=10000
[00:04:00]               │ debg TestSubjects.click(.slack-ActionTypeSelectOption)
[00:04:00]               │ debg Find.clickByCssSelector('[data-test-subj=".slack-ActionTypeSelectOption"]') with timeout=10000
[00:04:00]               │ debg Find.findByCssSelector('[data-test-subj=".slack-ActionTypeSelectOption"]') with timeout=10000
[00:04:01]               │ debg TestSubjects.click(addNewActionConnectorButton-.slack)
[00:04:01]               │ debg Find.clickByCssSelector('[data-test-subj="addNewActionConnectorButton-.slack"]') with timeout=10000
[00:04:01]               │ debg Find.findByCssSelector('[data-test-subj="addNewActionConnectorButton-.slack"]') with timeout=10000
[00:04:01]               │ debg TestSubjects.setValue(nameInput, 15b0e087906b4c1f8ea1a01c767057e7)
[00:04:01]               │ debg TestSubjects.click(nameInput)
[00:04:01]               │ debg Find.clickByCssSelector('[data-test-subj="nameInput"]') with timeout=10000
[00:04:01]               │ debg Find.findByCssSelector('[data-test-subj="nameInput"]') with timeout=10000
[00:04:01]               │ debg TestSubjects.setValue(slackWebhookUrlInput, https://test)
[00:04:01]               │ debg TestSubjects.click(slackWebhookUrlInput)
[00:04:01]               │ debg Find.clickByCssSelector('[data-test-subj="slackWebhookUrlInput"]') with timeout=10000
[00:04:01]               │ debg Find.findByCssSelector('[data-test-subj="slackWebhookUrlInput"]') with timeout=10000
[00:04:01]               │ debg Find.clickByCssSelector('[data-test-subj="saveActionButtonModal"]:not(disabled)') with timeout=10000
[00:04:01]               │ debg Find.findByCssSelector('[data-test-subj="saveActionButtonModal"]:not(disabled)') with timeout=10000
[00:04:01]               │ debg Find.findByCssSelector('.euiToast') with timeout=20000
[00:04:02]               │ debg Find.findByCssSelector('.euiToastHeader__title') with timeout=10000
[00:04:02]               │ debg Find.clickByCssSelector('.euiToast__closeButton') with timeout=10000
[00:04:02]               │ debg Find.findByCssSelector('.euiToast__closeButton') with timeout=10000
[00:04:02]               │ debg Find.findByCssSelector('[data-test-subj="messageTextArea"]') with timeout=10000
[00:04:02]               │ debg TestSubjects.setValue(messageTextArea, test message )
[00:04:02]               │ debg TestSubjects.click(messageTextArea)
[00:04:02]               │ debg Find.clickByCssSelector('[data-test-subj="messageTextArea"]') with timeout=10000
[00:04:02]               │ debg Find.findByCssSelector('[data-test-subj="messageTextArea"]') with timeout=10000
[00:04:03]               │ debg TestSubjects.click(messageAddVariableButton)
[00:04:03]               │ debg Find.clickByCssSelector('[data-test-subj="messageAddVariableButton"]') with timeout=10000
[00:04:03]               │ debg Find.findByCssSelector('[data-test-subj="messageAddVariableButton"]') with timeout=10000
[00:04:03]               │ debg TestSubjects.click(variableMenuButton-0)
[00:04:03]               │ debg Find.clickByCssSelector('[data-test-subj="variableMenuButton-0"]') with timeout=10000
[00:04:03]               │ debg Find.findByCssSelector('[data-test-subj="variableMenuButton-0"]') with timeout=10000
[00:04:05]               │ debg TestSubjects.click(messageAddVariableButton)
[00:04:05]               │ debg Find.clickByCssSelector('[data-test-subj="messageAddVariableButton"]') with timeout=10000
[00:04:05]               │ debg Find.findByCssSelector('[data-test-subj="messageAddVariableButton"]') with timeout=10000
[00:04:05]               │ debg TestSubjects.click(variableMenuButton-1)
[00:04:05]               │ debg Find.clickByCssSelector('[data-test-subj="variableMenuButton-1"]') with timeout=10000
[00:04:05]               │ debg Find.findByCssSelector('[data-test-subj="variableMenuButton-1"]') with timeout=10000
[00:04:05]               │ debg TestSubjects.click(saveAlertButton)
[00:04:05]               │ debg Find.clickByCssSelector('[data-test-subj="saveAlertButton"]') with timeout=10000
[00:04:05]               │ debg Find.findByCssSelector('[data-test-subj="saveAlertButton"]') with timeout=10000
[00:04:05]               │ debg Find.findByCssSelector('.euiToast') with timeout=20000
[00:04:07]               │ debg Find.findByCssSelector('.euiToastHeader__title') with timeout=10000
[00:04:07]               │ debg Find.clickByCssSelector('.euiToast__closeButton') with timeout=10000
[00:04:07]               │ debg Find.findByCssSelector('.euiToast__closeButton') with timeout=10000
[00:04:07]               │ debg TestSubjects.find(alertSearchField)
[00:04:07]               │ debg Find.findByCssSelector('[data-test-subj="alertSearchField"]') with timeout=10000
[00:04:08]               │ debg Find.findByCssSelector('.euiBasicTable[data-test-subj="alertsList"]:not(.euiBasicTable-loading)') with timeout=10000
[00:04:08]               │ debg Find.findByCssSelector('[data-test-subj="alertsList"] table') with timeout=10000
[00:04:08]               └- ✓ pass  (11.0s) "Actions and Triggers app create alert should create an alert"
[00:04:08]             └-> should create an alert with actions in multiple groups
[00:04:08]               └-> "before each" hook: global before each
[00:04:08]               │ debg Find.findByCssSelector('[data-test-subj="createAlertButton"],[data-test-subj="createFirstAlertButton"]') with timeout=10000
[00:04:08]               │ debg TestSubjects.setValue(alertNameInput, 4cd043b8806e44c3b33b86488ff22afd)
[00:04:08]               │ debg TestSubjects.click(alertNameInput)
[00:04:08]               │ debg Find.clickByCssSelector('[data-test-subj="alertNameInput"]') with timeout=10000
[00:04:08]               │ debg Find.findByCssSelector('[data-test-subj="alertNameInput"]') with timeout=10000
[00:04:09]               │ debg TestSubjects.click(test.always-firing-SelectOption)
[00:04:09]               │ debg Find.clickByCssSelector('[data-test-subj="test.always-firing-SelectOption"]') with timeout=10000
[00:04:09]               │ debg Find.findByCssSelector('[data-test-subj="test.always-firing-SelectOption"]') with timeout=10000
[00:04:09]               │ debg TestSubjects.click(.slack-ActionTypeSelectOption)
[00:04:09]               │ debg Find.clickByCssSelector('[data-test-subj=".slack-ActionTypeSelectOption"]') with timeout=10000
[00:04:09]               │ debg Find.findByCssSelector('[data-test-subj=".slack-ActionTypeSelectOption"]') with timeout=10000
[00:04:10]               │ debg TestSubjects.click(addNewActionConnectorButton-.slack)
[00:04:10]               │ debg Find.clickByCssSelector('[data-test-subj="addNewActionConnectorButton-.slack"]') with timeout=10000
[00:04:10]               │ debg Find.findByCssSelector('[data-test-subj="addNewActionConnectorButton-.slack"]') with timeout=10000
[00:04:10]               │ debg TestSubjects.setValue(nameInput, f3381162f3b84fd0bfccd5a6192451aa)
[00:04:10]               │ debg TestSubjects.click(nameInput)
[00:04:10]               │ debg Find.clickByCssSelector('[data-test-subj="nameInput"]') with timeout=10000
[00:04:10]               │ debg Find.findByCssSelector('[data-test-subj="nameInput"]') with timeout=10000
[00:04:10]               │ debg TestSubjects.setValue(slackWebhookUrlInput, https://test)
[00:04:10]               │ debg TestSubjects.click(slackWebhookUrlInput)
[00:04:10]               │ debg Find.clickByCssSelector('[data-test-subj="slackWebhookUrlInput"]') with timeout=10000
[00:04:10]               │ debg Find.findByCssSelector('[data-test-subj="slackWebhookUrlInput"]') with timeout=10000
[00:04:10]               │ debg Find.clickByCssSelector('[data-test-subj="saveActionButtonModal"]:not(disabled)') with timeout=10000
[00:04:10]               │ debg Find.findByCssSelector('[data-test-subj="saveActionButtonModal"]:not(disabled)') with timeout=10000
[00:04:10]               │ debg Find.findByCssSelector('.euiToast') with timeout=20000
[00:04:11]               │ debg Find.findByCssSelector('.euiToastHeader__title') with timeout=10000
[00:04:11]               │ debg Find.clickByCssSelector('.euiToast__closeButton') with timeout=10000
[00:04:11]               │ debg Find.findByCssSelector('.euiToast__closeButton') with timeout=10000
[00:04:11]               │ debg TestSubjects.setValue(messageTextArea, test message )
[00:04:11]               │ debg TestSubjects.click(messageTextArea)
[00:04:11]               │ debg Find.clickByCssSelector('[data-test-subj="messageTextArea"]') with timeout=10000
[00:04:11]               │ debg Find.findByCssSelector('[data-test-subj="messageTextArea"]') with timeout=10000
[00:04:12]               │ debg Find.findByCssSelector('[data-test-subj="alertActionAccordion-0"] [data-test-subj="messageTextArea"]') with timeout=10000
[00:04:12]               │ debg TestSubjects.click(addAlertActionButton)
[00:04:12]               │ debg Find.clickByCssSelector('[data-test-subj="addAlertActionButton"]') with timeout=10000
[00:04:12]               │ debg Find.findByCssSelector('[data-test-subj="addAlertActionButton"]') with timeout=10000
[00:04:12]               │ debg TestSubjects.click(.slack-ActionTypeSelectOption)
[00:04:12]               │ debg Find.clickByCssSelector('[data-test-subj=".slack-ActionTypeSelectOption"]') with timeout=10000
[00:04:12]               │ debg Find.findByCssSelector('[data-test-subj=".slack-ActionTypeSelectOption"]') with timeout=10000
[00:04:12]               │ debg TestSubjects.setValue(messageTextArea, test message )
[00:04:12]               │ debg TestSubjects.click(messageTextArea)
[00:04:12]               │ debg Find.clickByCssSelector('[data-test-subj="messageTextArea"]') with timeout=10000
[00:04:12]               │ debg Find.findByCssSelector('[data-test-subj="messageTextArea"]') with timeout=10000
[00:04:13]               │ debg Find.findByCssSelector('[data-test-subj="alertActionAccordion-1"] [data-test-subj="messageTextArea"]') with timeout=10000
[00:04:14]               │ debg TestSubjects.click(addNewActionConnectorActionGroup-1)
[00:04:14]               │ debg Find.clickByCssSelector('[data-test-subj="addNewActionConnectorActionGroup-1"]') with timeout=10000
[00:04:14]               │ debg Find.findByCssSelector('[data-test-subj="addNewActionConnectorActionGroup-1"]') with timeout=10000
[00:04:14]               │ debg TestSubjects.click(addNewActionConnectorActionGroup-1-option-other)
[00:04:14]               │ debg Find.clickByCssSelector('[data-test-subj="addNewActionConnectorActionGroup-1-option-other"]') with timeout=10000
[00:04:14]               │ debg Find.findByCssSelector('[data-test-subj="addNewActionConnectorActionGroup-1-option-other"]') with timeout=10000
[00:04:14]               │ debg TestSubjects.click(saveAlertButton)
[00:04:14]               │ debg Find.clickByCssSelector('[data-test-subj="saveAlertButton"]') with timeout=10000
[00:04:14]               │ debg Find.findByCssSelector('[data-test-subj="saveAlertButton"]') with timeout=10000
[00:04:14]               │ debg Find.findByCssSelector('.euiToast') with timeout=20000
[00:04:16]               │ debg Find.findByCssSelector('.euiToastHeader__title') with timeout=10000
[00:04:16]               │ debg Find.clickByCssSelector('.euiToast__closeButton') with timeout=10000
[00:04:16]               │ debg Find.findByCssSelector('.euiToast__closeButton') with timeout=10000
[00:04:16]               │ debg TestSubjects.find(alertSearchField)
[00:04:16]               │ debg Find.findByCssSelector('[data-test-subj="alertSearchField"]') with timeout=10000
[00:04:17]               │ debg Find.findByCssSelector('.euiBasicTable[data-test-subj="alertsList"]:not(.euiBasicTable-loading)') with timeout=10000
[00:04:17]               │ debg Find.findByCssSelector('[data-test-subj="alertsList"] table') with timeout=10000
[00:04:17]               └- ✓ pass  (8.8s) "Actions and Triggers app create alert should create an alert with actions in multiple groups"
[00:04:17]             └-> should show save confirmation before creating alert with no actions
[00:04:17]               └-> "before each" hook: global before each
[00:04:17]               │ debg Find.findByCssSelector('[data-test-subj="createAlertButton"],[data-test-subj="createFirstAlertButton"]') with timeout=10000
[00:04:17]               │ debg TestSubjects.setValue(alertNameInput, 47c5ede154514ace8509c2f552927c6f)
[00:04:17]               │ debg TestSubjects.click(alertNameInput)
[00:04:17]               │ debg Find.clickByCssSelector('[data-test-subj="alertNameInput"]') with timeout=10000
[00:04:17]               │ debg Find.findByCssSelector('[data-test-subj="alertNameInput"]') with timeout=10000
[00:04:18]               │ debg TestSubjects.click(.index-threshold-SelectOption)
[00:04:18]               │ debg Find.clickByCssSelector('[data-test-subj=".index-threshold-SelectOption"]') with timeout=10000
[00:04:18]               │ debg Find.findByCssSelector('[data-test-subj=".index-threshold-SelectOption"]') with timeout=10000
[00:04:18]               │ debg TestSubjects.click(selectIndexExpression)
[00:04:18]               │ debg Find.clickByCssSelector('[data-test-subj="selectIndexExpression"]') with timeout=10000
[00:04:18]               │ debg Find.findByCssSelector('[data-test-subj="selectIndexExpression"]') with timeout=10000
[00:04:18]               │ debg Find.findByCssSelector('#indexSelectSearchBox') with timeout=10000
[00:04:28]               │ info Taking screenshot "/dev/shm/workspace/parallel/11/kibana/x-pack/test/functional/screenshots/failure/Actions and Triggers app create alert should show save confirmation before creating alert with no actions.png"
[00:04:29]               │ info Current URL is: http://localhost:61111/app/management/insightsAndAlerting/triggersActions/alerts
[00:04:29]               │ info Saving page source to: /dev/shm/workspace/parallel/11/kibana/x-pack/test/functional/failure_debug/html/Actions and Triggers app create alert should show save confirmation before creating alert with no actions.html
[00:04:29]               └- ✖ fail: Actions and Triggers app create alert should show save confirmation before creating alert with no actions
[00:04:29]               │      TimeoutError: Waiting for element to be located By(css selector, #indexSelectSearchBox)
[00:04:29]               │ Wait timed out after 10020ms
[00:04:29]               │       at /dev/shm/workspace/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
[00:04:29]               │       at runMicrotasks (<anonymous>)
[00:04:29]               │       at processTicksAndRejections (internal/process/task_queues.js:97:5)
[00:04:29]               │ 
[00:04:29]               │ 

Stack Trace

TimeoutError: Waiting for element to be located By(css selector, #indexSelectSearchBox)
Wait timed out after 10020ms
    at /dev/shm/workspace/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:97:5) {
  remoteStacktrace: ''
}

Metrics [docs]

Distributable file count

id before after diff
default 43202 43203 +1

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@MadameSheema MadameSheema merged commit 37e9070 into elastic:master Dec 3, 2020
@MadameSheema MadameSheema deleted the indicator-match branch December 3, 2020 10:48
MadameSheema added a commit to MadameSheema/kibana that referenced this pull request Dec 3, 2020
…ss test (elastic#84323)

* implemnts indicator match rule cypress test

* fixes merge issue

* fixes type check issues

* fixes mapping

* simplifies data

* fixes excpetions flakiness

* fixes alerts test

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
gmmorris added a commit to gmmorris/kibana that referenced this pull request Dec 3, 2020
* master: (40 commits)
  fix: 🐛 don't add separator befor group on no main items (elastic#83166)
  [Security Solution][Detections] Implements indicator match rule cypress test (elastic#84323)
  [APM] Add APM agent config options (elastic#84678)
  Fixed a11y issue on rollup jobs table selection (elastic#84567)
  [Discover] Refactor getContextUrl to separate file (elastic#84503)
  [Embeddable] Export CSV action for Lens embeddables in dashboard (elastic#83654)
  [TSVB] [Cleanup] Remove extra dateFormat props (elastic#84749)
  [Lens] Migrate legacy es client and remove total hits as int (elastic#84340)
  Improve logging pipeline in @kbn/legacy-logging (elastic#84629)
  Catch @hapi/podium errors (elastic#84575)
  [Discover] Unskip date histogram test (elastic#84727)
  Rename server.xsrf.whitelist to server.xsrf.allowlist (elastic#84791)
  [Enterprise Search] Fix schema errors button (elastic#84842)
  [APM] Removes react-sticky dependency in favor of using CSS (elastic#84589)
  [Maps] Always initialize routes on server-startup (elastic#84806)
  [Fleet] EPM support to handle uploaded file paths (elastic#84708)
  [Snapshot Restore] Fix initial policy form state (elastic#83928)
  Upgrade Node.js to version 14 (elastic#83425)
  [Security Solution] Keep Endpoint policies up to date with license changes (elastic#83992)
  [Security Solution][Exceptions] Implement exceptions for ML rules (elastic#84006)
  ...
gmmorris added a commit to gmmorris/kibana that referenced this pull request Dec 3, 2020
* master: (236 commits)
  fix: 🐛 don't add separator befor group on no main items (elastic#83166)
  [Security Solution][Detections] Implements indicator match rule cypress test (elastic#84323)
  [APM] Add APM agent config options (elastic#84678)
  Fixed a11y issue on rollup jobs table selection (elastic#84567)
  [Discover] Refactor getContextUrl to separate file (elastic#84503)
  [Embeddable] Export CSV action for Lens embeddables in dashboard (elastic#83654)
  [TSVB] [Cleanup] Remove extra dateFormat props (elastic#84749)
  [Lens] Migrate legacy es client and remove total hits as int (elastic#84340)
  Improve logging pipeline in @kbn/legacy-logging (elastic#84629)
  Catch @hapi/podium errors (elastic#84575)
  [Discover] Unskip date histogram test (elastic#84727)
  Rename server.xsrf.whitelist to server.xsrf.allowlist (elastic#84791)
  [Enterprise Search] Fix schema errors button (elastic#84842)
  [APM] Removes react-sticky dependency in favor of using CSS (elastic#84589)
  [Maps] Always initialize routes on server-startup (elastic#84806)
  [Fleet] EPM support to handle uploaded file paths (elastic#84708)
  [Snapshot Restore] Fix initial policy form state (elastic#83928)
  Upgrade Node.js to version 14 (elastic#83425)
  [Security Solution] Keep Endpoint policies up to date with license changes (elastic#83992)
  [Security Solution][Exceptions] Implement exceptions for ML rules (elastic#84006)
  ...
gmmorris added a commit to gmmorris/kibana that referenced this pull request Dec 3, 2020
…overy-action-group

* upstream/master: (48 commits)
  [Lens] accessibility screen reader issues (elastic#84395)
  [Logs UI] Fetch single log entries via a search strategy (elastic#81710)
  fix: 🐛 don't add separator befor group on no main items (elastic#83166)
  [Security Solution][Detections] Implements indicator match rule cypress test (elastic#84323)
  [APM] Add APM agent config options (elastic#84678)
  Fixed a11y issue on rollup jobs table selection (elastic#84567)
  [Discover] Refactor getContextUrl to separate file (elastic#84503)
  [Embeddable] Export CSV action for Lens embeddables in dashboard (elastic#83654)
  [TSVB] [Cleanup] Remove extra dateFormat props (elastic#84749)
  [Lens] Migrate legacy es client and remove total hits as int (elastic#84340)
  Improve logging pipeline in @kbn/legacy-logging (elastic#84629)
  Catch @hapi/podium errors (elastic#84575)
  [Discover] Unskip date histogram test (elastic#84727)
  Rename server.xsrf.whitelist to server.xsrf.allowlist (elastic#84791)
  [Enterprise Search] Fix schema errors button (elastic#84842)
  [APM] Removes react-sticky dependency in favor of using CSS (elastic#84589)
  [Maps] Always initialize routes on server-startup (elastic#84806)
  [Fleet] EPM support to handle uploaded file paths (elastic#84708)
  [Snapshot Restore] Fix initial policy form state (elastic#83928)
  Upgrade Node.js to version 14 (elastic#83425)
  ...
MadameSheema added a commit that referenced this pull request Dec 7, 2020
…ss test (#84323) (#84882)

* implemnts indicator match rule cypress test

* fixes merge issue

* fixes type check issues

* fixes mapping

* simplifies data

* fixes excpetions flakiness

* fixes alerts test

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:skip Skip the PR/issue when compiling release notes Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v7.11.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants