-
Notifications
You must be signed in to change notification settings - Fork 4.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update/e2e tests options #14129
Update/e2e tests options #14129
Conversation
@gziolo travis is failing again, would my change break anything other than this simple command? |
I've restarted the tests, failures appear unrelated to this PR |
I had to restart the one test again—passing now! |
This is consistent though with other usage / recommendations though: Lines 188 to 189 in fe2136d
Personally, if it's an argument, it feels more appropriate to use a standard arguments syntax. It feels a bit awkward initially, but this is the standard way of passing arguments to the underlying command of any npm script. Related: npm/npm#5518
It's an interesting point, since we have options supported by our abstraction, and options to pass through to Jest. Arguably, we should be explicit with our interface in such a way that we only pass through arguments either formed in our script handler, or whitelisted to be handled by Jest. But this is already at odds with how we use the test scripts, where as seen above we pass through Jest's I don't really have a good solution on mind at the moment. |
Yes, this is the way how it is handled with npm. At the same time you can always use:
and everything is going to work as expected. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It doesn't work the way it was proposed:
> gutenberg@5.1.1 test-e2e /Users/gziolo/PhpstormProjects/gutenberg
> wp-scripts test-e2e --config packages/e2e-tests/jest.config.js "interactive"
No tests found, exiting with code 1
Run with `--passWithNoTests` to exit with code 0
In /Users/gziolo/PhpstormProjects/gutenberg/packages/e2e-tests
250 files checked.
testMatch: **/__tests__/**/*.js,**/?(*.)(spec|test).js,**/test/*.js - 62 matches
testPathIgnorePatterns: /node_modules/ - 250 matches
testRegex: - 250 matches
Pattern: interactive - 0 matches
The issue is that interactive
is passed as a regular CLI param and Jest processes it as it was file pattern.
The only way to make it work is what @aduth shared:
Arguably, we should be explicit with our interface in such a way that we only pass through arguments either formed in our script handler, or whitelisted to be handled by Jest. But this is already at odds with how we use the test scripts, where as seen above we pass through Jest's
--updateSnapshot
and--watch
arguments.
This command integrates both Jest and Puppeteer so it becomes a bit complex in comparison to all other commands. One way of making it easier to whitelist would be to prefix all custom arguments with --puppeteer-
.
0d994e8
to
5fa1970
Compare
@gziolo I have implemented the argument as @aduth suggested with the extra dashes. Now the command can optionally be run as:
Also, I left the speed at 80 as a default, and also, see above, added a new argument for changing the speed. To solve the problems of passing all new arguments to Jest I added a small new function that removes arguments with certain prefixes from the list of arguments that is passed to Jest. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Another option would to not filter by prefix, but instead by command name. This way we would have shorter arguments for example:
npm run test-e2e:watch -- --interactive --slowmo=10
and in the cleanUpArgs function I could just take each one out.
Yes, that would work, too. I don't feel strongly about any of this.
…ment passing using double dash
…ment passing using double dash
17fe104
to
a50944e
Compare
@gziolo I have cleaned up the functions by moving the argument removal by prefix into the getCliArgs function of the process util. I've also rebased. We should be good to go :) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice one, we still need to document these new features. I will open a follow-up with more documentation improvements.
Awesome work @draganescu |
* added an interactive argument to the test-e2e script * mend * added a way to bypass certain args from Jest, reverted to normal argument passing using double dash * simplified the argument cleanup function * simplified the argument cleanup function * added a way to bypass certain args from Jest, reverted to normal argument passing using double dash * simplified the argument cleanup function * simplified the argument cleanup function * cleaned up the code by moving the prefixed arguments removal to getcliargs
* added an interactive argument to the test-e2e script * mend * added a way to bypass certain args from Jest, reverted to normal argument passing using double dash * simplified the argument cleanup function * simplified the argument cleanup function * added a way to bypass certain args from Jest, reverted to normal argument passing using double dash * simplified the argument cleanup function * simplified the argument cleanup function * cleaned up the code by moving the prefixed arguments removal to getcliargs
Description
Following #13993 and part of #8319 this adds the option to run all e2e test commands with an extra interactive commands, which shows Chrome at a human speed, so you can visually inspect your test.
So now we can run:
How has this been tested?
Ran locally with the interactive command and appears to work as expected.
Note
@talldan suggested we'd have
but the only way to have that is if we'd also append more '--' as in
and I figured it'd be too many dashes, plus when passed to jest it would throw an unrecognised arg error so we'd have clean it before.
Just using the word seemed to work out of the box, maybe ;)