Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Support for running Cypress e2e tests under different licenses #143932

Open
banderror opened this issue Oct 25, 2022 · 4 comments
Labels
Feature:Functional Testing Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. test_ui_functional

Comments

@banderror
Copy link
Contributor

Summary

We need to be able to test at the e2e level (Cypress tests) certain features or bugfixes that depend on the current license, for example:

Currently, we have support for different licenses in our API integration tests. Some are run under the Basic license, and some under the Trial one. But we don't have support for that in the Cypress tests.

Let's add this support. We can start with some research and building a PoC and later we can create more concrete tickets if needed.

@banderror banderror added test-coverage issues & PRs for improving code test coverage Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Rule Management Security Detection Rule Management Team labels Oct 25, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@MadameSheema
Copy link
Member

@banderror I did a research on this topic before with Threat Hunting team because they wanted to have some licensing tests on Cypress as well, they end up testing the behaviour on a different layer, so nothing was finally implemented. Here I'll try to list all my findings and useful information we need to take into consideration:

  • Our Cypress tests are executed on top of FTR.
  • By default when executing Cypress we are using the Trial license
  • Licenses work per instance
  • Once you change the Trial license you cannot return to that one.
  • Here is an example of how other teams are dealing with licensing in a different testing layer.

Taking into account all the above, at first glance, there are two solutions we can implement:

  1. Change the license each time that is required for a test.
  2. To have different cypress configurations, one per each license type.

Personally, I'm in favor of the second option, I'm not a fan of the first one because:

  • If for whatever reason the license is not correctly updated, other tests are prone to fail increasing flakiness and false failures.
  • We need to have the licenses hardcoded on the code, meaning we need to manually maintain the licenses.

Happy to provide a solution for this if needed or to help with the implementation.

@banderror
Copy link
Contributor Author

I need to think about that @MadameSheema, thank you so much for sharing this info!

@banderror banderror added test_ui_functional Feature:Functional Testing and removed test-coverage issues & PRs for improving code test coverage labels Mar 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Functional Testing Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. test_ui_functional
Projects
None yet
Development

No branches or pull requests

3 participants