Browser support

[mattdesl]

Trying to get this working in the browser, and then later hihat for debugging/breakpoints/etc.

After browserifying the demo in the readme, I get the following:

Uncaught ReferenceError: setImmediate is not defined

relevant line

(weird cause I thought browserify would have shimmed this)

[sindresorhus]

Maybe open an issue on Browserify about shimming setImmediate?

[mattdesl]

Found a bit more discussion upstream: browserify/insert-module-globals#40

[sindresorhus]

I'll give it a week, but I doubt anything will happen there...

I guess I can add https://github.com/sindresorhus/set-immediate-shim if that is the only thing blocking it from being used in the browser.

[Qix-]

Isn't it just a fancy way of doing setTimeout(fn, 0) or Window.requestAnimationFrame?

[sindresorhus]

@Qix- It's faster than setTimeout.

http://www.nczonline.net/blog/2013/07/09/the-case-for-setimmediate/

[tomek-he-him]

@vdemedes thanks for the fix! The setImmediate error is busted.

But now I’ve got another bastard there:

Uncaught TypeError: Cannot read property 'write' of undefined

– coming from https://github.com/kevva/squeak/blob/v1.2.0/index.js#L144.

[vadimdemedes]

@tomekwi You're welcome!

I see what the problem is, will try to not forget and fix it!

[tomek-he-him]

Great! I can’t wait to check out this awesome trio: hihat, ava and quixote!

quixote doesn’t go well with tape, because it’s an assertion library. So I end up with lots of glue code in tests and lose the pretty format of error messages.

[Qix-]

Another cool tool, quixote. Thanks for the link 💃

[mattdesl]

In the browser process.stdout doesn't exist and browserify doesn't try to shim it. Tape uses the following to print in the browser:
https://github.com/substack/tape/blob/master/lib/default_stream.js

[Qix-]

Shimming process.stdout would be simple, assuming somewhere along the way you write a newline.

[danielepolencic]

same issue with https://github.com/kevva/squeak/blob/v1.2.0/index.js#L144 :(

[sindresorhus]

Browserify should already shim process, so sounds like either a Browserify bug or an issue with your build setup.

[danielepolencic]

Browserify shims process, but looks like it doesn't shim process.stderr.

I think that causes https://github.com/kevva/squeak/blob/v1.2.0/index.js#L26 to be undefined and line 144 to fail.

[sindresorhus]

defunctzombie/node-process#35

I guess we can make squeak use console.log/console.error instead in the browser. PR welcome there.

[danielepolencic]

I tweaked the constructor and injected the console in squeak like you suggested.

This seems to fix the issue. I can run ava in browserify + hihat. 🎉 🎉

I get some errors about process.exit and bluebird, though.

ATM, I think it would be better to monkey patch process.stderr and process.exit rather than just injecting console in squeak - like you suggested in that link.

EDIT: maybe these patches should be done in hihat?

[sindresorhus]

Yeah. I guess we could replace the logger.js file with a custom logger-browser.js using the Browserify replace feature.

And process.exit = window.close;. https://github.com/Jam3/hihat#basic-examples

The process will stay open until you call window.close() from the client code.

[sindresorhus]

Happy to recommend hihat in the readme when we get this working :)

[mattdesl]

Added a ticket about exit: Experience-Monks/hihat#27

Not sure about shimming stderr/stdout in browserify. Seems like browserify's process module could provide a polyfill, but all the stream stuff could add a lot of extra bytes to bundles for mostly no gain.

[mattdesl]

With that said; maybe a dumb polyfill could just write console.log and error synchronously, but not attempt to support any other aspect of the stdout/stderr streams.

[danielepolencic]

This is the line that fails with the process.exit.

As for process.stderr I think the issue is not as straightforward as I thought.

ava IS browserify friendly (a part from process.exit), but squeak is not. In theory, the latter should be patched and not ava.

The issue with squeak is that even if I patch the stream to redirect to console, all output would be redirected to console.log and we lose all log types (i.e. error, warn, info, etc.). What it would be better is to patch squeak (or use another library) so that it falls back to console and still maintains all the right logging levels.

[sindresorhus]

What it would be better is to patch squeak (or use another library)

Agreed.

so that it falls back to console and still maintains all the right logging levels.

How would it do that? Prefix the console output with logging level?

[danielepolencic]

With the following (hacky) change in index.js:

var log = new Squeak({separator: ' ', stream: {write: console.log.bind(console)}});

all the logs from type success and error are redirected to console.log.

Ideally, one would like to have the following mappings between squeak and console

Squeak Type	logging level
info	console.info
success	console.log
error	console.error
warn	console.warn

winston offers such functionality (and multiple transports too), but it's not browserify friendly.

I think the quickest win in this case would be to use console directly?

as an example log.error could be emulated with:

var logError = console.error.bind(console, figures.cross);

That would work in node and in the browser.

[sindresorhus]

Shouldn't be hard to patch squeak to use console.log when used in the browser though. I think that would be the best and less hacky solution.

[danielkcz]

I am wondering if there is anything that I could do to help with this.

Personally I wouldn't even mind if those tests would not run in parallel. If I delegate test running to some service like Saucelabs and it will run all tests in series for each browser, it should be more than fine. Going for hassle with web workers just seems like overkill.

[jamestalmage]

Personally I wouldn't even mind if those tests would not run in parallel

I think that's a fine first goal, and if it gets us to browser support faster, so be it.

Going for hassle with web workers just seems like overkill.

We've identified other problems with web workers (see above), so that's definitely off the table.

That said, isolation is way more important than parallel execution. The two best solutions for isolation (WebWorkers and iframes) also facilitate parallel execution, so I'm not sure we need to punt on either for long.

I think iframes should work OK. We may need to come up with something to allow for apps that care about window.top, but we can punt on that for now.

I am wondering if there is anything that I could do to help with this

Reviewing and commenting on #887 is a good start. If you've got alternate ideas, proof-of-concept PR's wouldn't hurt.

[danielkcz]

I think iframes should work OK. We may need to come up with something to allow for apps that care about window.top, but we can punt on that for now.

Sounds to me like trying to cover for some eventuality that is actually rarely needed. Or at least I don't have any experience when I would need to rely on window.top.

In my case it's merely about having ability to run library unit tests in browser environment to ensure there isn't some incompatibility. As much as its nice to have tests written in same way, I wouldn't probably choose Ava for writing/running E2E tests as there are surely some other more mature tools for that.

I will have a look at that PR, thanks.

[fregante]

Yeah, surely iframe issues are rare, but if you never know what hacks polyfills and such have to rely on. Also what happens when two iframes try to interact with the History API? That's another one

[jamestalmage]

Hmm. Maybe we should start with running test files one at a time, top-level, launching a new browser/tab for each one. That will be really slow, but it's probably the easiest to accomplish. Then we can work towards parallelism.

[andrewmclagan]

That sounds like a splendid idea.

[novemberborn]

I'm curious what kind of browser support people are looking for.

• Do you want to see test results in the browser? In the CLI? Both?
• Do you start browser tests manually by navigating to a URL, or should AVA launch your browsers?
• How would you run browser tests in CI? Through SauceLabs / BrowserStack / others, or local VMs?
• Does your code need to be compiled specifically to be tested in a browser environment?

Please add anything else that seems relevant to the discussion.

[fregante]

I'd use this on Travis+SauceLabs to verify real compatibility with browsers, rather than "compatibility with jsdom." My issue is I have no experience in setting this up, so documentation specific to AVA+selenium would be great, perhaps that's the only thing missing.

[hoschi]

I give it a try with my current setup and use cases. On local (developer) machine tests get executed in jsdom environment for best performance. On CI server these tests should run in a real browser environment. If one of these tests fail, I want to be able to debug it with browsers dev tools.

Do you want to see test results in the browser? In the CLI? Both?

CLI or file to save it for each build.

Do you start browser tests manually by navigating to a URL, or should AVA launch your browsers?

Both is needed to run a specific test in a specific browser and automate the process in a whole?

How would you run browser tests in CI? Through SauceLabs / BrowserStack / others, or local VMs?

Not used these kind of tools for a long time, no real opinion here, but BroswerStack sounds good.

Does your code need to be compiled specifically to be tested in a browser environment?

As this are unit tests, I think dev compiled is ok. Productivity compiled code is for integration tests, I would say. Test code/setup should be configurable like the "--require" flag already does, at the moment I setup jsdom with this flag.

[gajus]

Can the title of this issue be set to "Browser support". The "Browserify" part makes it sound off-topic.

[sindresorhus]

Done

[elaijuh]

I am finding a way to do angular 2 unit test. All I could find is karma + jasmine + browser launcher. What I expect is I could use AVA to tackle it, so this issue looks like working on providing a browser environment for client side code.

I have tried browser-env with no luck working together with zone.js, client side support is a solid thing to me.

[mAAdhaTTah]

I'm currently playing around with using AVA to unit test a framework I'm working on (wooo Observables!), but without being able to run the unit tests themselves in the browser, it's a non-starter. I'm definitely interested in helping out; I played around with karma-ava but couldn't get it working.

Is there any movement on this, or anything I can look at or help out with?

[novemberborn]

@mAAdhaTTah we're still evolving AVA's internals too much to really commit to browser support, sorry.

[wearhere]

I'm curious what kind of browser support people are looking for.

To be really specific, the ideal outcome for Mixmax would be for Ava to work with the Karma test runner. What Karma lets you do is take the same test files that you would run in a Node environment, and run them using real browsers or devices or PhantomJS or headless Chrome instances. Karma launches the browser, and logs results to the CLI (at least how we use it); and as far as CI goes, headless Chrome works just fine, without special configuration, on Travis. The tests do need to be compiled for the browser i.e. bundle any dependencies (including Node modules) into a single IIFE or load them as globals.

We see that there is a Karma plugin for Ava here but it says "alpha level" currently, not sure if that's blocked on anything in this project.

(Want to try out Karma? See installation and configuration instructions here, though we can't vouch for those, we use/maintain https://github.com/mixmaxhq/erik i.e. currently use Jasmine for running unit tests in the browser, Ava only for server tests—though it would be nice to use Ava everywhere!)

[novemberborn]

Hey all, it's hopefully become apparent since this issue was first opened that browser support is not a priority for AVA. I am therefore closing this issue.

We are interested in letting you use AVA to drive browser tests, through Karma or other solutions. If you've got experience with this we'd love to see updates to the browser testing recipe, or indeed new recipes.

If you want to help out integrating browser tests by building on top of AVA then please get in touch. We don't have the resources to spearhead these efforts ourselves, but we'd love to include mature solutions in our GitHub and npm organizations, or promote them in recipes. Forking karma-ava might be a good start.

Thank you all for your interest in AVA and participation in this issue ❤️

[bj0]

sorry to bump an old issue, but I just tried using ava for the first time to test some client side js that uses Blob and SubtleCrypto apis, but they are not defined i node (I guess? I've never used node...).

Is the only way to use these APIs to run in browser?

[novemberborn]

@bj0 you'll have to stub those APIs in Node.js. Have a look at our browser testing recipe — the libraries mentioned there may already implement these APIs. (Or not, I haven't actually checked for you, sorry.)

[fabiospampinato]

Hello there, for anybody interested in this I'm kind of rewriting ava for browser support: https://github.com/fabiospampinato/fava, the thing currently has many drawbacks compared to ava, but it's designed to work in the browser, so if you need that maybe give it a try, if you find issues with it I'll be happy to address them.

It's not quite the same as just running ava itself in the browser, but the APIs are largely the same so you can switch back to ava once that's implemented here if you like.

[nicholascelestin]

Hello There,

For anyone interested in running ava tests in a browser, I've got a way to do that documented in this PR: #3086 . Basically, you run ava in a node process like usual, and a macro delegates execution of the test code to a headless Puppeteer browser.