Running tests only for a module?

[dmitriz]

I would like to run TDD tests only for a selected module. Is there any Grunt task for it?

Development instructions in
https://github.com/angular-ui/bootstrap/wiki/Development
only mention grunt watch, which is massive and slow.

For specific modules, only build:module is mentioned,
(whereas watch:module ignores the module),
with further instructions somewhat sketchy:
"Check the Grunt build file for other tasks that are defined for this project."

I couldn't tell how to do it from looking to Gruntfile.js but I'm no "Grunt expert" (using Gulp instead, which I prefer by far).

[chrisirhc]

You can edit karma.conf.js to include files only from the modules you need. Otherwise, you can modify the tests to use fdescribe for the particular tests you want to run.

[dmitriz]

I was hoping for a cleaner solution, don't want getting merge conflicts when pulling updates from upstream.

I would prefer to use a separate custom karma config but not sure how to make it run with Grunt.

There are also subtleties when modifying karma:

• one does not forget the dependencies of modules, e.g. bindHtml and position for typeahead:

• the following elegant brace matching globs will NOT watch for file changes (see feat(watcher): Allow using braces in watcher karma-runner/karma#1251, which is not fixed after all):

  // list of files / patterns to load in the browser
  files: [
     ...
    'src/{typeahead,bindHtml,position}/**/*.js',
    'template/{typeahead,bindHtml,position}/**/*.js'
  ],
  

One has to use the more verbose globs without braces instead.

[dmitriz]

The other solution with fdescribe unfortunately doesn't work (without additional hacking) because fdescribe breaks the validators:

$ grunt watch
Running "enforce" task

Running "ddescribe-iit:files" (ddescribe-iit) task
>> src/typeahead/test/typeahead.spec.js has `fdescribe` at line 1
Warning: Task "ddescribe-iit:files" failed. Use --force to continue.

Aborted due to warnings.

[dmitriz]

See working config in this commit:

test(typeahead): add custom config for typeahead

Custom Karma configuration to only run tests for typeahead module
autoWatch is switched to true, so Karma will rerun all tests upon file changes
run as: karma start karma-custom-typeahead.conf.js

https://github.com/dmitriz/bootstrap/commit/f5e4dcbf009945a566142e33e97f3daab4b0eb30

[chrisirhc]

To get around the ddescribe validation, just use --force like:

grunt watch --force
# or
grunt --force

I don't think we should change anything in the code to support this as we do want all the tests to run by default.

Perhaps it's a good idea to add these instructions to the wiki.

[dmitriz]

Thanks, it is better :)

But still not ideal.

When I am working on specific module, I want to keep running all related tests, not just ones I changed to fdescribe. Some related tests are spread over several files. Manually editing them back and forth just doesn't feel right.

[dmitriz]

Concerning running all tests...

It takes 25 seconds (!) for the whole run on my Macbook Pro (8 GB RAM, 1600 MHz DDR3), I don't want to think how slow it is on cheaper less powerful machines.

Worse - I get 3 full screens of warnings not related to the module I am working on - do you really want to see them all after every single edit?

But wait - it gets even worse!
As I type and save, your first edit is picked by Karma and tests are run. During the next 25++ seconds as they run, Karma will ignore all you following edits. What that effectively means, you will end up with untested potentially faulty code but will still see "Done, without errors."!
You will see the errors eventually but possibly too many at a time, which can easily drive you (at least me) insane 😱

[dmitriz]

Here I submitted a related PR:
#3548

[chrisirhc]

You can set fdescribe on multiple describe blocks. I don't think this should go into core. If you're working on a module, it's is a temporary state. I don't think the core should support your use case specifically. We want to always want to run all tests all the time since everything should be working.

As I said before, you can feel free to edit karma.conf.js locally. You don't need to commit it, you should in fact leave it unstaged so you know that you're setting up a custom environment just for working on that module. In that future, you want to reset that state.

If anything, we should instead spend time and effort on making the tests run faster.

[pkozlowski-opensource]

Agreed with @chrisirhc here. And we should really remove fdescribe checks on watch - we only care that are not left behind in the final build - having them during watch is OK.

[chrisirhc]

Ah right, indeed, it makes sense to remove those fdescribe checks on each watch run. It can run it when it's not a watch statement or during the initial run but not on each subsequent watch run.

[chrisirhc]

Hm. I'm running it on my machine and I don't see the warnings on every run. I only see it on the initial run. Perhaps @dmitriz has a different configuration?

[dmitriz]

I have created a Gist to demonstrate the testing output:
https://gist.github.com/dmitriz/6bdf94d41f6c652bd15f

By changing reportSlowerThan: inside karma.conf.js I can suppress the warnings.
Note that time can slightly change for different test runs, so suppressed warnings can also change.

With reportSlowerThan: 300 all warnings disappear.

But the problem is not the warnings, it is the 25 seconds to complete tests on each run

I may be missing something but I can't understand how that can be practical.

Is everyone doing TDD and working on the code, indeed running all 700+ tests upon each file save?

[chrisirhc]

I'm using fdescribe when developing specific modules.

[wesleycho]

It should be noted that fdescribe and fit can be used for convenience in combination with grunt karma:continuous or karma start karma.conf.js. This avoids the whole re-running tests problem while working in an isolated area of the codebase.