Show test duration

[colutti]

Is there any way to show how long the execution of all tests took? When using --verbose, AVA only shows the time when the tests finished.

[jamestalmage]

On unix systems you can use the time command:

time ava

Windows makes it way more complicated. If you need Windows support, you might be better off using gulp-ava

[colutti]

Thank you.

The gulp-ava link redirects me to my own question. Cant this be implemented in AVA instead?

[jamestalmage]

The gulp-ava link redirects me to my own question

Sorry. Writing markdown on mobile is not fun. https://github.com/avajs/gulp-ava

Can't this be implemented in AVA instead?

That idea has already been rejected. Test run times are highly impacted by background tasks, our caching mechanisms, concurrency, etc. It would be a fairly useless metric.

[fearphage]

Test run times are highly impacted by background tasks, our caching mechanisms, concurrency, etc. It would be a fairly useless metric.

Your statement applies to individual test timings as well. Are those an equally useless metric? Should I submit a PR to remove them?

[jordanh]

Adding 0.02 here. Our product is using the new CircleCI 2.0 beta. They provide an interface for collecting test timing information and providing statistics. You can use the information to configure CircleCI's test parallelization.

If I submitted a PR for timing, even if it's variable, would it be rejected?

[novemberborn]

@jordanh could you expand on how that interface works? What information specifically do you require out of AVA?

[jordanh]

Sure! It's a pretty simple. CircleCI expects a directory full of JUnit-formatted XML files (boy, do we ever need a better standard than this, but I'll reserve from making further judgments). The XML looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="rspec" tests="727" failures="0" errors="0" time="417.957921" timestamp="2017-01-27T14:53:40+00:00">
  <properties/>
  <testcase classname="spec.helpers.breadcrumbs.breadcrumbs_helper_spec" name="Breadcrumbs::BreadcrumbsHelper returns a breadcrumbs array" file="./spec/helpers/breadcrumbs/breadcrumbs_helper_spec.rb" time="1.797002"/>
...
</testsuite>

Today, I get real close to this format by using --tap and emitting XUnit by piping ava to tap-xunit.

There are a couple of routes to making this work with ava:

1. Adding XUnit/JUnit output directly to ava, and include: test suite start time, test suite duration, and test case execution duration

2. Provide the same information as 1, but thread the timing information through on the TAP interface in YAML payloads

Option (1) is more practical, I think, because there is no standard format for providing timing information in the TAP spec.

[novemberborn]

Given that AVA executes tests concurrently, both inside a worker process and across worker processes we think there's too much variability in execution time on the test case or even test suite level. It might average out OK for the entire test run (all test files and tests).

I'm not sure how we organize "suites" in our TAP output. I personally wouldn't be opposed to adding timestamps for the "global test suite" (being all test files and tests) in our TAP output, but not at any more granular level.

[jordanh]

Could we view this as a simple map/reduce pattern? map() over testcases, reduce() over testsuites.

May I look into a patch and come back with real data vs. speculation?

[novemberborn]

May I look into a patch and come back with real data vs. speculation?

Of course. It's incredibly unlikely though that we'll expose time measurements on the individual test level, and even doing it for the total execution time may not like. Just trying to give you a fair warning 😄

[novemberborn]

We're now open to including test duration in TAP output but only if --serial is used. Please see #1668 if you're willing to help us implement this.

I ended up using timecmd ava for windows, and placed timecmd.bat in the root folder of the project. The bat file contains https://stackoverflow.com/a/6209392/670839