Log test duration for serial tests only

[novemberborn]

I'm thinking of revisiting #971, but with a twist.

Reporting test duration when tests run concurrently can be misleading, since slow, synchronous code in one test also slows down another. This is why I've been hesitant of exposing the duration numbers in the TAP reporter.

Oddly though we do show durations in the verbose reporter! This has survived all the way from #30.

I wonder if we should only show durations when --serial is used. This way the numbers will be more accurate and users will be less tempted to fix one "slow" test while the problem is another. We can then also include it in the TAP output.

[sindresorhus]

👍

[oantoro]

Hello @novemberborn I am interested in working on this issue.

My question is, how should we format the output?

TAP version 13
# passing
ok 1 - passing 30 ms

1..1
# tests 1
# pass 1
# fail 0

is it okay?

[novemberborn]

@okyantoro yay!

I don't know — the spec is at http://testanything.org/tap-version-13-specification.html.

This may require a PR to https://www.npmjs.com/package/supertap which we use to format the TAP output.

[oantoro]

@novemberborn that spec didn't mention about time duration but I think it is reasonable to put the time duration information in test description.

I think it is okay to add duration options in supertap.test()
we will have something like this

supertab.test('Test Foo()', { ..., duration: test.duration, ... });

but if we concat the test title with test duration inside ava, we don't need to modify the supertab behavior, we also already have pretty-ms as dependency to format time duration.

[oantoro]

It is harder than it seem, directly passing the title and duration together like

TAP version 13
# passing 30ms
ok 1 - passing 30ms

1..1
# tests 1
# pass 1
# fail 0

The above output will not work at least if we pipe those output with tap-spec and tap-summary.
When we try to format those output with tap-summary, for example $ cat test.txt | npx tap-summary the result would be:

# Tests

✔ unnamed test [pass: 1, fail: 0, duration: 4ms]

# Summary

duration: 4ms
planned: 1
assertions: 1
pass: 1
fail: 0

or maybe we can put the time duration in supertap comment option?

[novemberborn]

It should work with tap-spec, tap-summary etc. Perhaps have a look at what other test runners do, e.g. node-tap.