Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FTR] add service to test user roles on serverless #170417

Merged
merged 83 commits into from
Dec 4, 2023

Conversation

dmlemeshko
Copy link
Member

@dmlemeshko dmlemeshko commented Nov 2, 2023

Summary

This PR enables user roles testing in FTR

We use SAML authentication to get session cookie for user with the specific role. The cookie is cached on FTR service side so we only make SAML auth one time per user within FTR config run. For Kibana CI service relies on changes coming in #170852

In order to run FTR tests locally against existing MKI project:

  • add .ftr/role_users.json in Kibana root dir
{
  "viewer": {
    "email": "...",
    "password": "..."
  },
  "developer": {
    "email": "...",
    "password": "..."
  }
}

  • set Cloud hostname (!not project hostname!) with TEST_CLOUD_HOST_NAME, e.g.
    export TEST_CLOUD_HOST_NAME=console.qa.cld.elstc.co

How to use:

  • functional tests:
const svlCommonPage = getPageObject('svlCommonPage');

before(async () => {
  // login with Viewer role  
  await svlCommonPage.loginWithRole('viewer');
  // you are logged in in browser and on project home page, start the test 
});

it('has project header', async () => {
  await svlCommonPage.assertProjectHeaderExists();
});
  • API integration tests:
const svlUserManager = getService('svlUserManager');
const supertestWithoutAuth = getService('supertestWithoutAuth');
let credentials: { Cookie: string };

before(async () => {
  // get auth header for Viewer role  
 credentials = await svlUserManager.getApiCredentialsForRole('viewer');
});

it('returns full status payload for authenticated request', async () => {
    const { body } = await supertestWithoutAuth
    .get('/api/status')
    .set(credentials)
    .set('kbn-xsrf', 'kibana');

    expect(body.name).to.be.a('string');
    expect(body.uuid).to.be.a('string');
    expect(body.version.number).to.be.a('string');
});

Flaky-test-runner:

#1 https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4081
#2 https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4114

@dmlemeshko dmlemeshko added v8.12.1 release_note:skip Skip the PR/issue when compiling release notes FTR v8.12.0 v8.11.1 and removed v8.12.1 labels Nov 8, 2023
@dmlemeshko dmlemeshko self-assigned this Nov 13, 2023
Copy link
Member

@pheyos pheyos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
Also tested running against MKI ✔️

@dmlemeshko
Copy link
Member Author

Within @elastic/appex-qa we agreed to merge it on Monday morning.

@dmlemeshko
Copy link
Member Author

Flaky test runner spotted instability with 28/150 failures, all with the same issue: cookie was set, user profile still displaying old test_serverless user instead of viewer.
I found out that previous test suite is not doing the logout, that might be the root cause. Since we can't gurantee all the tests suite to have logout, I added cookies removal in loginWithRole 8cc5da6
Verify a stability fix with flaky test runner build https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4140#_ 🔄

@dmlemeshko
Copy link
Member Author

Another flaky-test-runner to validate the last commit https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4159

Copy link
Member

@pheyos pheyos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Latest changes LGTM, also checked on MKI ✔️

@dmlemeshko dmlemeshko enabled auto-merge (squash) December 4, 2023 17:54
@dmlemeshko dmlemeshko merged commit d75103e into elastic:main Dec 4, 2023
35 checks passed
@kibana-ci
Copy link
Collaborator

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @dmlemeshko

@kibanamachine kibanamachine added the backport:skip This commit does not require backporting label Dec 4, 2023
MadameSheema added a commit that referenced this pull request Dec 13, 2023
…ing (#172655)

Relates to:
* #166340
* #170852
* #170417
* #172678

## Summary

In this PR we are using the code implemented on
#170417 and
#172678 to allow SAML and role
testing inside Cypress.

* We are creating a Cypress task to use the above-developed code and be
able to retrieve a session cookie given a role.
* We updated the login task to know how we should perform the login
depending if we are in Serverless (MKI or serverless FTR) or ESS
* In the parallel serverless script:
* We are updating the `BASE_ENV_URL` variable to use the proper QA
environment (pending to be done in follow-up PRs, to extract this value
so it is not hardcoded cc @dkirchan )
* We are adding the `IS_SERVERLESS` environment variable needed for the
logic on the login task. This changed implied to update the
`es_archiver` file to continue work as expected.
* We have added the `TEST_CLOUD_HOST_NAME` environment variable needed
for the code we are reusing to retrieve the session cookie for MKI.
* We have updated the Security Solution quality gate script to set the
`role_users.json` file needed by the code we are reusing to get the
different session cookies on MKI
* We have adjusted the tests because the username now follows the
pattern `test <role>` (@dmlemeshko is it possible to have as username
just the role? Is this something that can impact other tests and teams?)
* We have [skipped](#173168) a
test that got unstable after the changes.

## How to test it in your machine

### Serverless FTR

1. Navigate to `x-pack/test/security_solution_cypress`
2. Execute `yarn cypress:open:qa:serverless`
3. Click on `E2E testing`
4. Click on any test to execute it


### Serverless MKI

Setup a valid Elastic Cloud API key for QA environment:

1. Navigate to QA environment.
2. Click on the `User menu button` located on the top right of the
header.
3. Click on `Organization`.
5. Click on the `API keys` tab.
6. Click on `Create API key` button.
7. Add a name, set an expiration date, assign an organization owner
role.
8. Click on `Create API key`
9. Save the value of the key

Store the saved key on `~/.elastic/cloud.json` using the following
format:

```json
{
  "api_key": {
    "qa": "<API_KEY>"
  }
}
```

Store the email and password of the account you used to login in the QA
Environment at the root directory of your Kibana project on
`.ftr/role_users.json`, using the following format:

```json
{
  "admin": {
    "email": "<email>",
    "password": "<password>"
  }
}
```

If you want to execute a test with a role different from the default
one, make sure you have created the user under your organization and is
added to the above json following the format:

```json
{
  "admin": {
    "email": "<email>",
    "password": "<password>"
  },
  "<roleName>": {
    "email": "<email>",
    "password": "<password>"
  }
}
```

1. Navigate to `x-pack/test/security_solution_cypress`
2. Execute `yarn cypress:open:qa:serverless`
3. Click on `E2E testing`
4. Click on any test to execute it

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting FTR release_note:skip Skip the PR/issue when compiling release notes v8.12.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants