Selenium testing (E2E, Functional Testing)

[joscha]

Has anyone looked into using the React Storybook for selenium testing? The well-defined stories make it ideal to run tests against them, the only obstacle I see is potentially the iframe, which makes writing steps cumbersome. Has anyone tried this before, potentially directly hitting the iframe.html and/changing the static output?

[arunoda]

Actually, you can hit the iframe directly. Here's how.
See:

• Here's the iframe: http://kadira-samples.github.io/react-button/?selectedKind=Button&selectedStory=some%20emojies%20as%20the%20text
• Here's the iframe: http://kadira-samples.github.io/react-button/iframe.html?selectedKind=Button&selectedStory=some%20emojies%20as%20the%20text&dataId=something

You can change the selectedKind and selectedStory query params to change stories. You must set the dataId query params. The value could be anything.

[travi]

I've been considering something similar lately. I've been thinking about extending our testing scope to go beyond unit tests and include some level of integration tests. The tests would likely align very closely with the scenarios that we have built stories for. Coordinating the two concepts could greatly improve automated verification, but still give us the benefit of live demos that we already love.

However, I feel like selenium would be awfully heavy for this type of a setup. Its really simple to render React to a string, so it would be quite easy to use a combination of tools like enzyme or cheerio to analyze the rendered output and make assertions with something like chai.

I'm very interested to see where everyone's thoughts go on this one and would be happy to help it evolve if I can.

[arunoda]

This is how we do this.

We use stories to test the UI components visually. We also use a enzyme setup to test functional testing.

In our case, both has different type of scenarios where you don't need some of visual (storybook) test scenarios for functional testing and wise versa.

So, we've two npm scripts like this:

npm run storybook
npm run ui-functional-tests:watch

I talk more about this here.

[travi]

@arunoda I agree with most of the points that you highlight in your article, but I think it comes down to scope of the tests. There are different approaches to testing depending on the scope intended to be covered by certain testing layers.

In my case, we unit test each component in isolation as well as have a testing layer similar to your description of functional tests at the application level. However, when larger components are composed of several smaller components, there is value in also testing their integration with each other.

This level of testing would primarily test a combination of pure components, meaning the tests would mostly be a verification that a certain set of inputs produces the expected details in the HTML as output. The key to this goal is that the tests would look to ensure that inputs within certain boundaries would result in putting the component into various states. These states end up mostly being the same states that we are already using storybook to test visually, but I would love to not need to duplicate all of the same scenarios in a separate tool in order to automate verification that we can accomplish from inspecting the HTML output.

In short, while visual testing of these stories is important, there can be big value to asserting certain aspects of the output automatically of the same scenarios that can be accomplished simply from inspecting the HTML output of each story. If there would be a way to enable this through storybook, I think it could be of significant value.

[arunoda]

Got it. I think we could provide a way to give access to the DOM and then we can write assertions. Is that something you are looking for?

[travi]

Thats the main idea. The output of renderToString() or even renderToStaticMarkup() should be enough to start with. However, I could see growing into something along the lines of what could be done with enzyme's mount().

I'm not sure what the api would end up looking like, though. I would expect it would turn out to be something like passing a callback that would expect the receive the html string and enable assertions to be made against that string. Reporting failures would obviously be important.

I'm not sure where the callback would be passed though. If just using .add() to add stories, it could maybe be an optional argument there, but what if you are using an add-on like addWithInfo()? Maybe this should be an add-on, but can you use multiple add-ons at the same time?

Finally, I would hope that running the tests could be done with a more direct script than starting the storybook instance.

These are just some thoughts that come to mind. Hopefully it helps shape any thoughts you might have.

[arunoda]

Great. @mnmtanish any input on this.

[thani-sh]

Yes, using an addon seems like the best idea we have so far.
sorry about the previous comment, I totally missed the point there.

[travi]

A few things that I'm curious about further thoughts on if this were an add-on

• would that require that the server instance was started in order to run the "tests"? i'm thinking we could approach this in a way that the story definitions are run with some separate processor that focuses on the tests without setting things up for browser rendering
• would that work if another add-on, like the info add-on, was being used for a story? i haven't seen examples of decorating a story with more than one add-on

[travi]

https://github.com/mthuret/storybook-addon-specifications covers my needs in this area quite well. I'm super excited to have that available.

As far as I'm concerned, this issue could be closed now.

[arunoda]

Awesome.
I think we'll have more tools in this area in the future as well.