Public API for start/end of all tests run

[amitzur]

Not sure this is a documentation issue, but the feature does exist in Cypress.
There's a need for listening to an event that fires when tests start running. I see 3 cases where this can happen when running cypress open:

1. Launch browser
2. Click on "run all tests" button
3. Reload the browser tab

When running cypress run this should happen only once, of course.

From inspecting the events that Cypress emits, I see run:start and run:end answer exactly this need. So the question is, can these be used as a public API for this hook? If so, I was right to think it's a documentation issue.

[brian-mann]

If you want to do things in the browser you can have root level before and after hooks which come from mocha and already provide this.

For plugin related events you have before:browser:launch, but there is no "end" events. In interactive mode we will not provide these because interactive mode cypress open never really "ends".

In run mode cypress run you can use the module API and when all the tests finish it will yield you a results object with everything. So today you already have several different options for programmatically tapping into before / after tests run in Cypress.

[amitzur]

I essentially wanted to be sure my code runs before any test-related code, but I can go with your suggestion to use root level before and after.

I'd argue that in interactive mode there is an end, and that is the finish point of all the tests that currently run. It's true that this is fluid when open during development.

My issue now is that in order to use root level after I need it to be able to fail the whole test in case there's a failure there. That doesn't happen today (as opposed to mocha), as I described here:
cypress-io/cypress#2296

[amitzur]

I'm reopening this since root level before and after hook doesn't answer my need. I'm not sure if Cypress changed behavior between v2 and v3. I suspect parallelism might have affected it.
When running cypress run, every spec file is being run separately and root level before and after hooks that were defined in the supportsFile happen once for each file.

There is a need to have a way to run a hook one time at the beginning and one time at the end of the whole run of all the files

[jennifer-shehane]

@amitzur Yes, this behavior did change in the 3.0 release - that before and after run for each spec file.

Perhaps our Module API would suit your needs better. You can run any code before the cypress.run() and any code after in the resolved Promise.

[amitzur]

I cannot use the module API since we run Cypress in a straight forward way and I wouldn't want to complicate that.
I do think this is an important hook, and can be achieved by providing plugins with the following API:

module.exports = (on, config) => {
  on('run:start', () => {...});
  on('run:end', () => {...});
}

By running localStorage.debug=* I noticed the events run:start and run:end provide this functionality. So I think the road to providing this functionality is short.

[jennifer-shehane]

Hey @amitzur, we agree. In fact the work for adding this hook has mostly been done and will be part of a future release. You can see those changes in these doc updates here: #890

[amitzur]

Thanks, that's great to know!
I only wonder if run:before and run:after are going to be available also in interactive mode. I believe there's no reason for them not to.

[jennifer-shehane]

@amitzur No, those would not be available in interactive mode. The run technically never 'ends' when in interactive mode since the browser stays open.

[gilgold]

Is there any update on this issue? I could really use those events right about now.. (all other solutions do not work / not relevant to my needs)

[jennifer-shehane]

@gilgold This issue is about documentation, not the Cypress product itself. See this issue for any actual events feature: cypress-io/cypress#2840

I'm going to close this documentation issue - as the behavior of before and `after has been documented correctly now. The rest of the comments seem related to actually wanting new features in Cypress covered in cypress-io/cypress#2840