Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make TAP reporter TAP13-capable #3552

Merged
merged 84 commits into from
Nov 10, 2018
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
d3c2df9
added test to make sure first line of report is version line
mollstam Aug 8, 2018
c067a6e
reporter will now print version line as first line
mollstam Aug 8, 2018
9c78b61
Changed test to look for plan (1..N) on second line.
mollstam Aug 8, 2018
78ab6c5
Added integration tests to make sure we conform to spec.
mollstam Aug 8, 2018
b59af19
Extended integration test to validate plan, test is growing...
mollstam Aug 8, 2018
ddc5b17
removed some section comments and minor restructure
mollstam Aug 8, 2018
f030a4b
broke out testLineMatcher to testLinePredicate with minor refactor.
mollstam Aug 8, 2018
43c94c8
added a bunch of predicates to identify lines
mollstam Aug 8, 2018
8d369b7
stacks and errors are now wrapped inside YAML blocks
mollstam Aug 8, 2018
9677744
changed arrow function to ye olde function to appease bots
mollstam Aug 8, 2018
3047441
found linter and made code conform, `npm run test` now clean.
mollstam Aug 8, 2018
253d76a
added reporters.fixture.js for testing reporter capabilities
mollstam Aug 9, 2018
e00b99c
Fixed error message potentially breaking YAML block, some cleanup.
mollstam Aug 9, 2018
1e559b3
TAP now inherits from Base
mollstam Aug 9, 2018
419fc33
TAP no longer inherits from Base, decided not to use any Base functio…
mollstam Aug 9, 2018
3551db3
TAP unit tests now expect message starting with literal
mollstam Aug 9, 2018
68d8b9c
TAP integration tests use reporters fixture instead of uncaught
mollstam Aug 9, 2018
3b5f88f
refactored yaml indentation to clean things up
mollstam Sep 19, 2018
3cac7f1
fix to refactored indent which added an extra space on some lines
mollstam Sep 19, 2018
b5b88a1
Retrigger appveyor test
mollstam Sep 19, 2018
ce24d96
TAP reporter inherits Base
mollstam Sep 20, 2018
d5b519d
TAP reporter uses runner.stats rather than internal tracking
mollstam Sep 20, 2018
3242408
added support for `spec` reporterOption and started implementing `TAP…
mollstam Sep 20, 2018
61e8c79
TAP13 tests now run with proper spec selected
mollstam Sep 20, 2018
8dc5b65
extracted fail reporting to producers
mollstam Sep 20, 2018
4b11a31
added missing tests to TAP12 spec which are identical to TAP13 expect…
mollstam Sep 20, 2018
38b8ce3
extracted pending reporting to producers
mollstam Sep 20, 2018
7951bf0
extracted pass reporting to producers
mollstam Sep 20, 2018
ff80cf4
extracted end reporting to producers and refactored to common base
mollstam Sep 20, 2018
0496ef3
fixed bad redeclare
mollstam Sep 20, 2018
42035e7
fix to integrations tests to use TAP13 spec
mollstam Sep 20, 2018
4d9188f
Merge branch 'reporter-tap-v13' of https://github.com/mollstam/mocha …
plroebuck Nov 4, 2018
03e3440
feat(reporters/tap.js): Make TAP reporter conform to TAP13
plroebuck Nov 4, 2018
b060a14
test(reporters.spec.js,tap.spec.js): Updated to match `tapVersion` re…
plroebuck Nov 4, 2018
76a74bd
added test to make sure first line of report is version line
mollstam Aug 8, 2018
7e72ff9
reporter will now print version line as first line
mollstam Aug 8, 2018
ce0eecd
Changed test to look for plan (1..N) on second line.
mollstam Aug 8, 2018
9352709
Added integration tests to make sure we conform to spec.
mollstam Aug 8, 2018
f027dc5
Extended integration test to validate plan, test is growing...
mollstam Aug 8, 2018
5d88454
removed some section comments and minor restructure
mollstam Aug 8, 2018
425b03f
broke out testLineMatcher to testLinePredicate with minor refactor.
mollstam Aug 8, 2018
2af2e84
added a bunch of predicates to identify lines
mollstam Aug 8, 2018
f6cf3cf
stacks and errors are now wrapped inside YAML blocks
mollstam Aug 8, 2018
308b381
changed arrow function to ye olde function to appease bots
mollstam Aug 8, 2018
009daed
found linter and made code conform, `npm run test` now clean.
mollstam Aug 8, 2018
ccbd91e
added reporters.fixture.js for testing reporter capabilities
mollstam Aug 9, 2018
b2746b3
Fixed error message potentially breaking YAML block, some cleanup.
mollstam Aug 9, 2018
9cb8990
TAP now inherits from Base
mollstam Aug 9, 2018
54ca9f7
TAP no longer inherits from Base, decided not to use any Base functio…
mollstam Aug 9, 2018
56752f1
TAP unit tests now expect message starting with literal
mollstam Aug 9, 2018
4680170
TAP integration tests use reporters fixture instead of uncaught
mollstam Aug 9, 2018
e75408d
refactored yaml indentation to clean things up
mollstam Sep 19, 2018
ab0db06
fix to refactored indent which added an extra space on some lines
mollstam Sep 19, 2018
f00ca30
TAP reporter inherits Base
mollstam Sep 20, 2018
ae782b8
TAP reporter uses runner.stats rather than internal tracking
mollstam Sep 20, 2018
9e2e5bf
added support for `spec` reporterOption and started implementing `TAP…
mollstam Sep 20, 2018
1b06541
TAP13 tests now run with proper spec selected
mollstam Sep 20, 2018
709f567
extracted fail reporting to producers
mollstam Sep 20, 2018
c646564
added missing tests to TAP12 spec which are identical to TAP13 expect…
mollstam Sep 20, 2018
4442207
extracted pending reporting to producers
mollstam Sep 20, 2018
f4590ee
extracted pass reporting to producers
mollstam Sep 20, 2018
552091b
extracted end reporting to producers and refactored to common base
mollstam Sep 20, 2018
dd1ebc1
fixed bad redeclare
mollstam Sep 20, 2018
cb0258d
fix to integrations tests to use TAP13 spec
mollstam Sep 20, 2018
f472b6b
3483: Squelch CI Growl-related spawn errors (#3517)
plroebuck Oct 24, 2018
207ed8c
test(test/reporters/base.spec.js): Fix Base reporter test failure due…
plroebuck Oct 20, 2018
02d75ee
Update "commander" to correct display of falsy default values (#3529)
outsideris Oct 27, 2018
ff6cbe7
refactor(bin/options.js): Refactor and improve documentation (#3533)
plroebuck Oct 30, 2018
2f28b49
refactor(json-stream.js): Consistent output stream usage (#3532)
plroebuck Oct 30, 2018
2d89a47
fix(_mocha): Update '--no-timeouts' argument description (#3546)
plroebuck Nov 2, 2018
f2f79c0
build(ESLint/Git): Ignore JSDoc output directory (#3544)
plroebuck Nov 2, 2018
ee4625f
ci(Travis/Appveyor): Update Node versions in CI matrix (#3543)
plroebuck Nov 2, 2018
cebb18a
update release instructions [ci skip]
boneskull Nov 2, 2018
9286fdc
fix runner to emit start/end event (#3395)
outsideris Nov 2, 2018
d4808a3
Warn that suites cannot return a value (#3550)
plroebuck Nov 4, 2018
c6c784a
feat(reporters/tap.js): Make TAP reporter conform to TAP13
plroebuck Nov 4, 2018
c179a34
test(reporters.spec.js,tap.spec.js): Updated to match `tapVersion` re…
plroebuck Nov 4, 2018
df03a00
test(reporters/helpers.js): Add update to "helpers.js" from PR #3528
plroebuck Nov 6, 2018
6d32ec9
test(reporters/tap.spec.js): Update TAP reporter test before merge
plroebuck Nov 6, 2018
3b01085
Merge branch 'mollstam-reporter-tap-v13' of https://github.com/mochaj…
plroebuck Nov 6, 2018
9d8b18a
test(reporters.tap.spec.js): Remove unused variable
plroebuck Nov 6, 2018
b3ea5ed
Merge branch 'master' into mollstam-reporter-tap-v13
plroebuck Nov 6, 2018
0ce70f8
docs(reporters/tap.js): Updated JSDoc
plroebuck Nov 10, 2018
e5b516e
Merge branch 'mollstam-reporter-tap-v13' of https://github.com/mochaj…
plroebuck Nov 10, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
264 changes: 235 additions & 29 deletions lib/reporters/tap.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,10 @@
* Module dependencies.
*/

var util = require('util');
var Base = require('./base');
var inherits = require('../utils').inherits;
var sprintf = util.format;

/**
* Expose `TAP`.
Expand All @@ -15,65 +18,268 @@ var Base = require('./base');
exports = module.exports = TAP;

/**
* Initialize a new `TAP` reporter.
* Constructs a new `TAP` reporter instance.
*
* @public
* @class
* @memberof Mocha.reporters
* @extends Mocha.reporters.Base
* @api public
* @param {Runner} runner
* @memberof Mocha.reporters
* @param {Runner} runner - Instance triggers reporter actions.
* @param {Object} [options] - runner options
*/
function TAP(runner) {
Base.call(this, runner);
function TAP(runner, options) {
Base.call(this, runner, options);

var self = this;
var n = 1;
var passes = 0;
var failures = 0;

runner.on('start', function() {
var total = runner.grepTotal(runner.suite);
console.log('%d..%d', 1, total);
var tapVersion = '12';
if (options && options.reporterOptions) {
if (options.reporterOptions.tapVersion) {
tapVersion = options.reporterOptions.tapVersion.toString();
}
}

this._producer = createProducer(tapVersion);

runner.once('start', function() {
var ntests = runner.grepTotal(runner.suite);
self._producer.writeVersion();
self._producer.writePlan(ntests);
});

runner.on('test end', function() {
++n;
});

runner.on('pending', function(test) {
console.log('ok %d %s # SKIP -', n, title(test));
self._producer.writePending(n, test);
});

runner.on('pass', function(test) {
passes++;
console.log('ok %d %s', n, title(test));
self._producer.writePass(n, test);
});

runner.on('fail', function(test, err) {
failures++;
console.log('not ok %d %s', n, title(test));
if (err.message) {
console.log(err.message.replace(/^/gm, ' '));
}
if (err.stack) {
console.log(err.stack.replace(/^/gm, ' '));
}
self._producer.writeFail(n, test, err);
});

runner.once('end', function() {
console.log('# tests ' + (passes + failures));
console.log('# pass ' + passes);
console.log('# fail ' + failures);
self._producer.writeEpilogue(runner.stats);
});
}

/**
* Return a TAP-safe title of `test`
* Inherit from `Base.prototype`.
*/
inherits(TAP, Base);

/**
* Returns a TAP-safe title of `test`.
*
* @api private
* @param {Object} test
* @return {String}
* @private
* @param {Test} test - Test instance.
* @return {String} title with any hash character removed
*/
function title(test) {
return test.fullTitle().replace(/#/g, '');
}

/**
* Writes newline-terminated formatted string to reporter output stream.
*
* @private
* @param {string} format - `printf`-like format string
* @param {...*} [varArgs] - Format string arguments
*/
function println(format, varArgs) {
var vargs = Array.from(arguments);
vargs[0] += '\n';
process.stdout.write(sprintf.apply(null, vargs));
}

/**
* Returns a `tapVersion`-appropriate TAP producer instance, if possible.
*
* @private
* @param {string} tapVersion - Version of TAP specification to produce.
* @returns {TAPProducer} specification-appropriate instance
* @throws {Error} if specification version has no associated producer.
*/
function createProducer(tapVersion) {
var producers = {
'12': new TAP12Producer(),
'13': new TAP13Producer()
};
var producer = producers[tapVersion];

if (!producer) {
throw new Error(
'invalid or unsupported TAP version: ' + JSON.stringify(tapVersion)
);
}

return producer;
}

/**
* @summary
* Constructs a new `TAPProducer` instance.
*
* @description
* Should <em>only</em> be used as an abstract base class.
*
* @private
* @constructor
*/
function TAPProducer() {}

/**
* Writes the TAP version to reporter output stream.
*
* @abstract
*/
TAPProducer.prototype.writeVersion = function() {};

/**
* Writes the plan to reporter output stream.
*
* @abstract
* @param {number} ntests - Number of tests that are planned to run.
*/
TAPProducer.prototype.writePlan = function(ntests) {
println('%d..%d', 1, ntests);
};

/**
* Writes that test passed to reporter output stream.
*
* @abstract
* @param {number} n - Index of test that passed.
* @param {Test} test - Instance containing test information.
*/
TAPProducer.prototype.writePass = function(n, test) {
println('ok %d %s', n, title(test));
};

/**
* Writes that test was skipped to reporter output stream.
*
* @abstract
* @param {number} n - Index of test that was skipped.
* @param {Test} test - Instance containing test information.
*/
TAPProducer.prototype.writePending = function(n, test) {
println('ok %d %s # SKIP -', n, title(test));
};

/**
* Writes that test failed to reporter output stream.
*
* @abstract
* @param {number} n - Index of test that was skipped.
* @param {Test} test - Instance containing test information.
* @param {Error} err - Reason the test failed.
*/
TAPProducer.prototype.writeFail = function(n, test, err) {
println('not ok %d %s', n, title(test));
};

/**
* Writes the summary epilogue to reporter output stream.
*
* @abstract
* @param {Object} stats - Object containing run statistics.
*/
TAPProducer.prototype.writeEpilogue = function(stats) {
// :TBD: Why is this not counting pending tests?
println('# tests ' + (stats.passes + stats.failures));
println('# pass ' + stats.passes);
// :TBD: Why are we not showing pending results?
println('# fail ' + stats.failures);
};

/**
* @summary
* Constructs a new `TAP12Producer` instance.
*
* @description
* It will produce output conforming to the TAP12 spec.
*
* @private
* @constructor
* @extends TAPProducer
* @see {@link https://testanything.org/tap-specification.html|Specification}
*/
function TAP12Producer() {
/**
* Writes that test failed to reporter output stream, with error formatting.
* @override
*/
this.writeFail = function(n, test, err) {
TAPProducer.prototype.writeFail.call(this, n, test, err);
if (err.message) {
println(err.message.replace(/^/gm, ' '));
}
if (err.stack) {
println(err.stack.replace(/^/gm, ' '));
}
};
}

/**
* Inherit from `TAPProducer.prototype`.
*/
inherits(TAP12Producer, TAPProducer);

/**
* @summary
* Constructs a new `TAP13Producer` instance.
*
* @description
* It will produce output conforming to the TAP13 spec.
*
* @private
* @constructor
* @extends TAPProducer
* @see {@link https://testanything.org/tap-version-13-specification.html|Specification}
*/
function TAP13Producer() {
/**
* Writes the TAP version to reporter output stream.
* @override
*/
this.writeVersion = function() {
println('TAP version 13');
};

/**
* Writes that test failed to reporter output stream, with error formatting.
* @override
*/
this.writeFail = function(n, test, err) {
TAPProducer.prototype.writeFail.call(this, n, test, err);
var emitYamlBlock = err.message != null || err.stack != null;
if (emitYamlBlock) {
println(indent(1) + '---');
if (err.message) {
println(indent(2) + 'message: |-');
println(err.message.replace(/^/gm, indent(3)));
}
if (err.stack) {
println(indent(2) + 'stack: |-');
println(err.stack.replace(/^/gm, indent(3)));
}
println(indent(1) + '...');
}
};

function indent(level) {
return Array(level + 1).join(' ');
}
}

/**
* Inherit from `TAPProducer.prototype`.
*/
inherits(TAP13Producer, TAPProducer);
63 changes: 63 additions & 0 deletions test/integration/fixtures/reporters.fixture.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
'use strict';

/**
* This file generates a wide range of output to test reporter functionality.
*/

describe('Animals', function() {

it('should consume organic material', function(done) { done(); });
it('should breathe oxygen', function(done) {
// we're a jellyfish
var actualBreathe = 'nothing';
var expectedBreathe = 'oxygen';
expect(actualBreathe, 'to equal', expectedBreathe);
done();
});
it('should be able to move', function(done) { done(); });
it('should reproduce sexually', function(done) { done(); });
it('should grow from a hollow sphere of cells', function(done) { done(); });

describe('Vertebrates', function() {
describe('Mammals', function() {
it('should give birth to live young', function(done) {
var expectedMammal = {
consumesMaterial: 'organic',
breathe: 'oxygen',
reproduction: {
type: 'sexually',
spawnType: 'live',
}
};
var platypus = JSON.parse(JSON.stringify(expectedMammal));
platypus['reproduction']['spawnType'] = 'hard-shelled egg';

expect(platypus, 'to equal', expectedMammal);
done();
});

describe('Blue Whale', function() {
it('should be the largest of all mammals', function(done) { done(); });
it('should have a body in some shade of blue', function(done) {
var bodyColor = 'blueish_grey';
var shadesOfBlue = ['cyan', 'light_blue', 'blue', 'indigo'];
expect(bodyColor, 'to be one of', shadesOfBlue);

done();
});
});
});
describe('Birds', function() {
it('should have feathers', function(done) { done(); });
it('should lay hard-shelled eggs', function(done) { done(); });
});
});

describe('Tardigrades', function() {
it('should answer to "water bear"', function(done) { done(); });
it('should be able to survive global mass extinction events', function(done) {
throw new Error("How do we even test for this without causing one?")
done();
});
});
});
Loading