-
-
Notifications
You must be signed in to change notification settings - Fork 697
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Asserting that assertions were made #94
Comments
This seems a bit tricky as a single line of an assertion can actually have multiple "assertions" (calls to |
QUnit has expect:
|
I'd have to dig into the code to familiarize myself with it before suggesting implementations. As for the actual assertion count, I'd say it's less important than knowing whether any assertions were made at all. The former is useful for stats geeks I guess, but hardly useful for any type of code quality measurements. However, tests that don't have any assertions what so ever need to be singled out, as they are by definition pretty useless in that case :) |
I like the idea of counts, just cause I'm a stats nerd. But the issue we run into is that QUnit's counting works because (1) the assertions and the test runner are integrated (2) the assertions aren't chain-able. A cheap and dirty way to do this as a plugin would have this interface... var chai = require('chai')
, chaiCount = require('chai-counter')
, should = chai.should();
chai.use(chaiCount);
var c = chai.counter();
something.should.equal('hello universe').count(c);
otherThing.should.have.property('hello')
.and.be.a('string').count(c);
c.assert(2); The count has to be the last in the chain in order for it to count properly. If it isn't last, there is no guarantee that the full thread finished. I'm sure there might be a better way to do this... |
:) My use case is a little bit different. I want to do this on a very global level, where before I run any test, the assertion counter (or flag) is reset, and after the test has run the counter/flag will be checked to see if it was hit at all. I use chai with mocha, and it's very easy to hook into before and after each test, but well in there I have no idea what to do with chai. I suppose solving that problem will inherently give you a correct counter as well, since you'd have to know whenever any type of assertion were made. Am I making any sort of sense? I feel like I might be confusing things. Edit: The benefit of having a global counter or flag is of course that I can then make sure to fail any tests that didn't make any assertions at all. It happens a little bit too often that I misspell the code and mocha happily reports these tests as A-OK, which is clearly not right. |
The only way I can think of a global counter/flag is to hook into Alternatively, if you were doing something like If you know all of chai's internals completely, the counting solution isn't too bad. But I don't think that should be a requisite. So, rock/hard place... |
Rock/hard place indeed. I think though, for my current use case, that hooking into |
There should be sufficient information on writing helper on the docs site to get you started, but feel free to post code for review here if you need further assistance. Also, when you have a solution you like, please post it as a Gist and include your information on the wiki page for helpers. Thank you kindly! |
You could hook into |
This thread seems to have died, but I have a need for this. I'm more interested in if any assertions occurred than the exact number. If there's any interest, I can take a look and make a PR for discussion. Shall I go ahead? |
Sure, be interested to see what you come up with! |
I'm definitely interested in this. I've fallen victim to the problem outlined above where no assertions were made but the test passes. |
+1 |
The issue may be the hidden usage of property getters. Does anyone know of a library that rewrites these into actual functions, so the invocation of these properties will throw an error if misspelled? E.g. expect(foo).to.be.treu() should rightly err. |
tpyo is a proof-of-concept to show how proxy can be used to allow misspellings, perhaps there's inspiration to be had there. |
By the way - to count assertions (or rather groups of assertions) you can use checkmark - the tiny library I created to overcome problems of that kind. Not the most elegant solution -- a kind of a workaround -- but it's simple, works for me and might also work for you. |
I like @radkodinev's way if it were to be written as a Chai plugin. For instance: var arr = ["array"];
it("should make an assertion", function(done) {
// After one assertion is made, run `done`
expect(1).check(done);
// `.mark` will increment the internal counter above.
arr.should.have.length(1).mark;
}) |
FWIW I've since switched to must which solves all of this nonsense quite elegantly. |
I made a gist implementing checkmark for chai if anyone's interested. It should accomplish most of what OP wants when using it with the |
I'm going to keep this open. I think this is a reasonable enough add-on to the core, if someone comes up with a sensible solution. In the meantime, @sirlancelot if you fancy making your gist an proper plugin (with an npm/bower package), you could add it to our plugins list (just edit the plugins.js file in chai/chai-docs) |
I got around to making a full-fledged repository with linting: https://github.com/sirlancelot/chai-checkmark |
@sirlancelot chai-checkmark is useful, thanks. I've implemented it in najax tests to make sure all different HTTP request types are covered. My usage case is somewhat different than the described case. I would be interested to see more usage examples here, to show how test implementations could look. |
Just got hit by this, this should be fixed. I can't rely on test framework which can't guarantee all assertions have been run. I'm now trying this must thing. P.S I just opened an issue in Mocha to implement pluggable assertion counting in Mocha |
Is this still an issue? I think you can call assertions as methods now, which prevents typos, etc. |
I think I jumped the gun a bit earlier by saying a PR is wanted. It would be better to flesh out a design here before moving to making PRs |
I have a quick question. how does it know what the last link in the chain is? Because if you just do it by the last link in the chain and that every link in the chain is chainable or is valid then it would solve most of the issues example: expect(foo).to.be.true this will add the number to the assertions when true is called bc its the last in the chain. This may not fix the typo issue like true but it does get much closer to the problem. It's like the mark library but built into the last assertion. |
@dtracers I don't think it's possible for a chain to know what its last link is, or for a link to know that it's the last link in a chain. At least not without some extreme AST parsing. A consequence of enabling expressive chainable assertions is that there's nothing stopping someone from doing some pretty whacky things, such as splitting each link of a chain onto a separate line and then branching into multiple assertions. This is a valid test: it("hurts my soul", function () {
var link1 = expect(42);
var link2 = link1.to;
var link3a = link2.equal(42);
var link3b = link2.not;
var link4b = link3b.equal(43);
}); |
so basically the only way to really know is after each test you build a tree of all of the calls for that specific test then check leaf nodes of that built tree. But the problem is that chai isnt really integrated on the test level. |
It seems this feature was not actually implemented. At least I didn't find anything neither in the documentation nor in the tests showing how to do it. in
In the tests:
|
Hi, jest has an expect.assertions method that verifies that a certain number of assertions are called during a test. I was hoping there was a chai equivalent to this. I've been searching in google and couldn't find anything so far. |
In the meantime I've improved a bit my extension (in TypeScript):
and the usage:
|
@martin-g This won't have the expected behavior for any of the For those methods, Chai also runs an additional 2 assertions internally to assert that |
I've made a quick helper to assist with this, that doesn't involve hooking into beforeEach/afterEach. I only use it where I need it. In the example, it is wired up to
|
Is there a way to assert whether any chai assertions have been made?
We've been bitten a number of times by tests passing gloriously in Mocha, only to realize that it's because no assertions were made for whatever reason (a common one is the misspelling of assertions, such as
calledwith
instead ofcalledWith
. This would easily be caught by having a test hook that runs after each test to make sure that assertions were indeed made. It won't help with situations where some assertions are made and others are broken by misspelling or such, but in our experience it's been less common (if occurring at all.)If not possible, I'm happy to contribute, just let me know!
The text was updated successfully, but these errors were encountered: