Running angular-mocks tests concurrently

[karlhorky]

I've been playing around with testing Angular directives in the AVA concurrent test runner and I've run into issues with getting the tests running concurrently. It appears angular-mocks rewrites variables in the afterEach() method, causing property reference errors:

Partial test setup

import test from 'ava';

// ...

global.window.beforeEach = test.beforeEach;
global.window.afterEach = test.afterEach;
require('angular-mocks');

// ...

test('a', t => t.true(true));
test('b', t => t.true(true));

Command line output (annotated with comment lines)

$ ava --verbose

# beforeEach for "a" called
# beforeEach for "b" called
  ✔ a
  ✔ b
# afterEach for "a" called  <-- currentSpec set to null here
  ✖ afterEach for "b" failed with "Cannot read property '$injector' of null"

  1 test failed

  1. afterEach for "b"
  TypeError: Cannot read property '$injector' of null
    Test.fn (node_modules/angular-mocks/angular-mocks.js:2593:31)

Relevant code from angular-mocks

  (window.beforeEach || window.setup)(function() {
    // ...
    currentSpec = this;
  });

  (window.afterEach || window.teardown)(function() {
    var injector = currentSpec.$injector;

    // ...

    currentSpec.$injector = null;
    currentSpec.$modules = null;
    currentSpec.$providerInjector = null;
    currentSpec = null;

    // ...

  });

Rewriting the tests to execute serially works around this error (but loses the nice property of concurrency):

test.serial('a', t => t.true(true));
test.serial('b', t => t.true(true));

A full example including failing and working cases here:
https://gist.github.com/karlhorky/6e68101d8167be27e027

Is concurrent tests something that the angular team is interested in supporting? If not, are there any other workarounds for getting angular-mocks running concurrently?

angular.js v. 1.5.0
angular-mocks v. 1.5.0

[petebacondarwin]

We have no plans to support concurrent testing and so AVA is probably not something that is going to work with AngularJS.
If you wanted to make the tests more concurrent, you could consider sharding the test base and running multiple completely separate karma instances.

[karlhorky]

Hm, ok that's too bad. Concurrent tests can really speed up test runs.

I guess the reason for this is that effort on new things like this is being put into Angular 2 now?

[karlhorky]

Thanks for the suggestion for karma, but I'm trying a minimal testing setup (trying to avoid too many tools). AVA is both the test framework and the runner.

I've found a workaround for the issue to still maintain concurrency - running the window.afterEach || window.teardown only once as a test.after call (instead of test.afterEach):

- global.window.afterEach = test.afterEach;
+ global.window.afterEach = test.after;

[karlhorky]

@kentcdodds long shot but I've seen some of your projects have included AVA lately and I know you know Angular, do you have any input on this?

[kentcdodds]

I think if you try to retrofit Angular tests to work with ava, you're going to have a hard time I'm afraid. But I've never tried.

[kentcdodds]

Note, main reason for this is ava does not support running tests in the browser. And Angular 1 needs the browser APIs to be tested reliably.

You could, however, write many of your services in vanilla JS, and test those in isolation.....

[karlhorky]

I'm running Angular in jsdom to emulate the browser environment: https://gist.github.com/karlhorky/6e68101d8167be27e027

The issue is mainly just angular-mocks and its mutation of shared state (currentSpec) across multiple beforeEach and afterEach calls. But I guess it's not something that anyone has taken a shot at fixing before.

[gkalpak]

I'm not familiar with AVA, but is the problem really that angukar-mocks mutates shared state or the fact that this state (e.g. the injector is shared among specs ?

[karlhorky]

I'm still learning how AVA works as well, but in my first few directive unit tests, the shared state itself doesn't appear to cause problems in the concurrent tests. I'm still able to initialize the module and inject services and providers like usual. The tests seem to run just like mocha.

The issue appears to be the following sequence:

window.afterEach = test.afterEach;

beforeEach 'test 1' 
beforeEach 'test 2'
✔ test 1
✔ test 2
afterEach 'test 1'  <-- currentSpec set to null here
afterEach 'test 2'  <-- currentSpec.$injector referenced here (error)

When I set the window.afterEach function in the angular-mocks scope to be run after all tests (test.after, similar to Jasmine's afterAll), the dereferencing of currentSpec doesn't happen until the very end, and no property reference errors occur.

window.afterEach = test.after;

beforeEach 'test 1' 
beforeEach 'test 2'
✔ test 1
✔ test 2
after() <-- currentSpec set to null here

[karlhorky]

@sindresorhus @jamestalmage any feedback on running angular-mocks with AVA? The relevant part of angular-mocks is here:

angular.js/src/ngMock/angular-mocks.js

Lines 2565 to 2619 in 72b96ef

	var currentSpec = null,
	annotatedFunctions = [],
	isSpecRunning = function() {
	return !!currentSpec;
	};
	angular.mock.$$annotate = angular.injector.$$annotate;
	angular.injector.$$annotate = function(fn) {
	if (typeof fn === 'function' && !fn.$inject) {
	annotatedFunctions.push(fn);
	}
	return angular.mock.$$annotate.apply(this, arguments);
	};
	(window.beforeEach || window.setup)(function() {
	annotatedFunctions = [];
	currentSpec = this;
	});
	(window.afterEach || window.teardown)(function() {
	var injector = currentSpec.$injector;
	annotatedFunctions.forEach(function(fn) {
	delete fn.$inject;
	});
	angular.forEach(currentSpec.$modules, function(module) {
	if (module && module.$$hashKey) {
	module.$$hashKey = undefined;
	}
	});
	currentSpec.$injector = null;
	currentSpec.$modules = null;
	currentSpec.$providerInjector = null;
	currentSpec = null;
	if (injector) {
	injector.get('$rootElement').off();
	injector.get('$rootScope').$destroy();
	}
	// clean up jquery's fragment cache
	angular.forEach(angular.element.fragments, function(val, key) {
	delete angular.element.fragments[key];
	});
	MockXhr.$$lastInstance = null;
	angular.forEach(angular.callbacks, function(val, key) {
	delete angular.callbacks[key];
	});
	angular.callbacks.counter = 0;
	});

My setup: https://gist.github.com/karlhorky/6e68101d8167be27e027