feat: Support auth0 provider in cypress tests #1121

vishal423 · 2022-04-24T19:38:36Z

Closes #892

Support auth0 OIDC provider in cypress e2e tests
Refactor cypress authentication logic to move individual provider authentication logic in a separate function.

vishal423 · 2022-04-24T19:55:15Z

To run cypress tests with auth0 users, you can provide ADMIN and USER role credentials as environment variables:

export CYPRESS_adminUsername=
export CYPRESS_adminPassword=
export CYPRESS_userUsername=
export CYPRESS_userPassword=

mraible · 2022-04-25T21:56:11Z

I tried to install this with npm i -g "vishal423/generator-jhipster-svelte#cypress-auth0-support". This failed, so I went with the good ol' npm link. I then tried to create an app using the following JDL (from the project's README, with oauth2 for auth):

application {
    config {
        baseName SampleBlogApp,
        applicationType monolith,
        authenticationType oauth2,
        packageName tech.jhipster.samples.blog,
        prodDatabaseType postgresql,
        cacheProvider caffeine,
        buildTool maven
    }
    entities *
}

entity Blog {
    name String required minlength(3)
    handle String required minlength(2)
}

entity Tag {
    name String required minlength(3)
}

relationship ManyToOne {
    Blog{user(login)} to User
    Post{blog(name)} to Blog
}

relationship ManyToMany {
    Post{tag(name)} to Tag{entry}
}

paginate Tag with pagination

$ shipster jdl app.jdl

This fails with the following error:

Error: In the relationship between Post and Blog, Post is not declared.
Error while parsing applications and entities from the JDL Error: In the relationship between Post and Blog, Post is not declared.

Next, I tried the JDL I use for a lot of my demos:

shipster jdl blog-oauth2

This works. If I try to start the Docker containers with a shortcut I have, it fails.

Missing script: "ci:e2e:prepare"

My shortcut:

run='npm run ci:e2e:prepare'

I started Keycloak, the app with ./mvnw, and ran npm run e2e, just to make sure the default still works. 1 test failed and it seems to take forever (~9m).

       Spec                                              Tests  Passing  Failing  Pending  Skipped
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  admin/logger.spec.js                     00:17        7        7        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/blog/blog-create.spec.js        00:11        4        4        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/blog/blog-delete.spec.js        00:11        3        3        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/blog/blog-list.spec.js          00:09        3        3        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/blog/blog-update.spec.js        00:18        5        5        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/blog/blog-view.spec.js          00:07        2        2        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/post/post-create.spec.js        00:15        4        4        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/post/post-delete.spec.js        00:19        3        3        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/post/post-list.spec.js          00:18        5        5        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  entities/post/post-update.spec.js        00:26        5        4        1        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/post/post-view.spec.js          00:07        2        2        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/tag/tag-create.spec.js          00:10        4        4        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/tag/tag-delete.spec.js          00:10        3        3        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/tag/tag-list.spec.js            00:12        5        5        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/tag/tag-update.spec.js          00:12        5        5        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/tag/tag-view.spec.js            00:08        2        2        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  footer.spec.js                           00:03        1        1        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  home.spec.js                             00:07        3        3        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  navbar.spec.js                           00:14       10       10        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  routes.spec.js                           00:13        3        3        -        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✖  1 of 20 failed (5%)                      04:18       79       78        1        -        -

Execution time: 8 min. 48 s.

Maybe the slowness is caused by my M1?

vishal423 · 2022-04-26T03:39:40Z

Thanks @mraible for your time.

I tried to install this with npm i -g "vishal423/generator-jhipster-svelte#cypress-auth0-support"

It also didn't work on my local setup. It fails for various reasons (initially due to previous symbolic links created by the npm link command and later due to the absence of husky under the global node_modules directory). I didn't dig deep to make it work as it's not my usual flow to test out a change.

I then tried to create an app using the following JDL (from the project's README, with oauth2 for auth)

Yeah, that's an issue as it doesn't declare a Post entity and tries to use that in relationships. I will raise a ticket to track this issue.

If I try to start the Docker containers with a shortcut I have, it fails.

Yes, we don't generate those npm commands as part of svelte blueprint. The idea is to provide standard familiar command interface and let individual's tune or create system wide shortcuts as they feel appropriate.

1 test failed and it seems to take forever (~9m)

I can also see same failure on local. That's not related to OAuth and is due to a specific datatype. I will raise a separate ticket to track this issue as well.

mraible · 2022-04-26T04:03:58Z

Do you see the slowness? I can try in my Intel MBP tomorrow (Tuesday MDT) to see if I can reproduce.

vishal423 · 2022-04-26T05:17:32Z

I didn't measure the exact time on the M1 MacBook, but it's slower than my Intel Linux laptop. On Linux, the same e2e tests complete within 2 minutes

──────────────────┘
✖ 1 of 20 failed (5%) 01:21 79 78 1 - -

mraible · 2022-04-26T18:57:08Z

I tried running tests on my Intel Mac today.

$ node --version
v16.14.0

It did complete much faster (2 min. 57 s.). I did have to click "Quit" a number of times because Chrome thought it was still downloading stuff.

If I try to create the same app with JHipster 7.8.1 (jhipster jdl blog-oauth2) and run npm run e2e, everything passes. I believe this might be because the multi-select dropdown in Svelte contains checkboxes instead of being a simple <select>. It also finishes 3x faster and has no Quit dialogs.

    ✔  All specs passed!                        00:32       23       23        -        -        -

Execution time: 53 s.

One minor difference I noticed is the Svelte blueprint creates a .yo-rc.json with tabs instead of spaces. Is that intentional?

vishal423 · 2022-04-26T19:15:06Z

I generally execute tests with e2e:ci and same also being tested in our CI. I haven't noticed the Quit dialog before, and ideally that shouldn't happen as we don't download anything. I will try tomorrow to reproduce on both Mac and Linux.

I will not compare speed with main generator as we do things differently and also test suite is much broader than the main (79 vs 23). Though, many of those can be converted to Junits, I will defer that along or after #172

Svelte blueprint style uses tabs over spaces (across code) 😄

mraible · 2022-04-26T19:50:23Z

I tried running e2e tests after configuring my Svelte app to use Auth0. I only defined two environment variables:

export CYPRESS_adminUsername=XXX
export CYPRESS_adminPassword=YYY

Tests worked up until entities/tag/tag-list.spec.js and then they stopped working because of Auth0 rate limits. This used to be an issue in the main generator, then we moved to Cypress sessions that don't authenticate as much.

I tried running e2e tests against Auth0 using the same app (generated with the main generator) and all tests pass. The Quit dialog did popup once.

I like that the main generator only requires you to define two variables (CYPRESS_E2E_USERNAME and CYPRESS_E2E_PASSWORD) and I think their names are easier to remember.

These values override the credentials for admin users and regular users. I'm guessing you changed this for this blueprint because you wanted more fine-grained control?

vishal423 · 2022-04-27T18:24:07Z

In main generator, we don't test flows with USER role privilege, whereas, over here we do have a test that do some assertions on the Admin menu to be hidden for regular users. If you look at the test code, these variables are internally used as Cypress.env('userUsername'), Cypress.env('userPassword'). If you have a better name, then, do let me know.

Further, we do use cypress session api support and avoid re-login. You can see the usage under custom login command. I also tried with Auth0 and haven't encountered the rate limit myself. This could be due to n/w latency on my side that avoided the limit. Use of session API is a stop gap solution and will not work with a large e2e test suite.

mraible · 2022-04-27T19:08:18Z

These are all valid points. I think my main concern is I can't run shipster jdl blog-oauth2 and have all e2e tests pass with Keycloak.

As far as variable names, I think it's good to use uppercase so it's less confusing. Since Cypress and e2e are redundant, maybe we can use CYPRESS_USER_USERNAME and CYPESS_ADMIN_USERNAME (and CYPRESS_USER_PASSWORD / CYPRESS_ADMIN_PASSWORD)?

vishal423 · 2022-04-27T20:00:48Z

sure, lets wait for e2e issue to be resolved. I have updated CI tests to use the JDL causing issue.

I will create a separate issue to track the change in variable names.

vishal423 · 2022-04-29T03:58:33Z

@mraible , can you please test to verify the e2e fix?

mraible · 2022-04-29T04:41:14Z

Looks good! If I generate an app with shipster jdl blog-oauth2, all tests pass.

    ✔  All specs passed!                        03:34       79       79        -        -        -

Execution time: 7 min. 29 s.

If I switch it to use Auth0, most of the tests pass.

export CYPRESS_adminUsername=foo
export CYPRESS_adminPassword=bar
npm run e2e
...

  (Run Finished)


       Spec                                              Tests  Passing  Failing  Pending  Skipped
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✖  admin/logger.spec.js                     00:28        7        6        1        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/blog/blog-create.spec.js        00:13        4        4        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/blog/blog-delete.spec.js        00:12        3        3        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/blog/blog-list.spec.js          00:11        3        3        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/blog/blog-update.spec.js        00:15        5        5        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/blog/blog-view.spec.js          00:10        2        2        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/post/post-create.spec.js        00:14        4        4        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✖  entities/post/post-delete.spec.js        00:07        3        -        1        -        2 │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/post/post-list.spec.js          00:13        5        5        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/post/post-update.spec.js        00:17        5        5        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/post/post-view.spec.js          00:09        2        2        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/tag/tag-create.spec.js          00:11        4        4        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/tag/tag-delete.spec.js          00:12        3        3        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/tag/tag-list.spec.js            00:13        5        5        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/tag/tag-update.spec.js          00:13        5        5        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  entities/tag/tag-view.spec.js            00:10        2        2        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  footer.spec.js                           00:03        1        1        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  home.spec.js                             00:09        3        3        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  navbar.spec.js                           00:16       10       10        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  routes.spec.js                           00:18        3        3        -        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✖  2 of 20 failed (10%)                     04:24       79       75        2        -        2

mraible · 2022-04-29T04:49:42Z

I tried again with Auth0 and this time all tests pass!

    ✔  All specs passed!                        04:24       79       79        -        -        -

Execution time: 7 min. 36 s.

vishal423 · 2022-05-23T18:25:40Z

@pascalgrimaud, Bounty claimed https://opencollective.com/generator-jhipster/expenses/78245

pascalgrimaud · 2022-05-23T21:32:14Z

@vishal423 : approved

vishal423 added this to the 0.9.0 milestone Apr 24, 2022

vishal423 requested a review from mraible April 24, 2022 20:16

This was referenced Apr 26, 2022

Update the JDL example in the project README file #1126

Closed

Cypress update entity test fails on the many-to-one relationships field #1127

Closed

vishal423 mentioned this pull request Apr 27, 2022

Rename cypress username/password environment variables #1135

Closed

vishal423 and others added 3 commits April 29, 2022 08:56

feat: Support auth0 provider in cypress tests

0195cc0

Run npm audit fix

573e315

Fix vulnerability with undefined

e75292c

vishal423 force-pushed the cypress-auth0-support branch from 9b9a4c5 to e75292c Compare April 29, 2022 03:27

mraible approved these changes Apr 29, 2022

View reviewed changes

vishal423 merged commit 606a877 into jhipster:main Apr 29, 2022

vishal423 deleted the cypress-auth0-support branch April 29, 2022 05:29

mraible mentioned this pull request May 4, 2022

Cypress prompts you to Continue Download or Quit every so often jhipster/generator-jhipster#18536

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Support auth0 provider in cypress tests #1121

feat: Support auth0 provider in cypress tests #1121

vishal423 commented Apr 24, 2022

vishal423 commented Apr 24, 2022

mraible commented Apr 25, 2022

vishal423 commented Apr 26, 2022

mraible commented Apr 26, 2022

vishal423 commented Apr 26, 2022

mraible commented Apr 26, 2022

vishal423 commented Apr 26, 2022

mraible commented Apr 26, 2022

vishal423 commented Apr 27, 2022

mraible commented Apr 27, 2022

vishal423 commented Apr 27, 2022

vishal423 commented Apr 29, 2022

mraible commented Apr 29, 2022

mraible commented Apr 29, 2022

vishal423 commented May 23, 2022

pascalgrimaud commented May 23, 2022

feat: Support auth0 provider in cypress tests #1121

feat: Support auth0 provider in cypress tests #1121

Conversation

vishal423 commented Apr 24, 2022

vishal423 commented Apr 24, 2022

mraible commented Apr 25, 2022

vishal423 commented Apr 26, 2022

mraible commented Apr 26, 2022

vishal423 commented Apr 26, 2022

mraible commented Apr 26, 2022

vishal423 commented Apr 26, 2022

mraible commented Apr 26, 2022

vishal423 commented Apr 27, 2022

mraible commented Apr 27, 2022

vishal423 commented Apr 27, 2022

vishal423 commented Apr 29, 2022

mraible commented Apr 29, 2022

mraible commented Apr 29, 2022

vishal423 commented May 23, 2022

pascalgrimaud commented May 23, 2022