Add a "strict" parameter to xfail

[nicoddemus]

• Add bool type to addini
• I reviewed skipping.rst while adding the new docs, I think it is easier to use it as a reference while staying true to its original "tutorial" intent (a crude rendering can be seen here).
• I didn't mention anything about future plans of changing the default value of the strict parameter from False to True. Not sure if we want to do that or not.

Fix #1355

[RonnyPfannschmidt]

well done, i wish GH had a merge on pass button

[nicoddemus]

Thanks!

@rabbbit could you review as well, and perhaps even try it? Would hate for this to be merged only to discover after the release it still does not fit your needs and others.

[rabbbit]

Does github have any badge options I could give you? :) 👍

Yeah, it looks awesome. I wouldn't bother with the bool-parsing, and I have troubles understanding actual code, but functionality wise it looks perfect.

[RonnyPfannschmidt]

we should consider re-priming the xfail result to also handle setup/teardown xfails instead of just doing a fail on exit

i wonder if xpass in strict mode should always report to the terminal :)

[nicoddemus]

Missed the CHANGELOG, I will just push it directly to master

[nicoddemus]

Thanks for merging it!

we should consider re-priming the xfail result to also handle setup/teardown xfails instead of just doing a fail on exit

What do you mean by "re-priming"?

i wonder if xpass in strict mode should always report to the terminal :)

The current implementation fails the test entirely... or do you mean something else?

[RonnyPfannschmidt]

what i meant is that

a) instead of running fail in the pyfunc_call, we should let a normal xfail propagate
b) interpret xfails based on the strict value, and always report strict ones,
while showing non-strict ones only if requested

[nicoddemus]

Just realized this is the wrong hook for this. 😦

It is not executing the test at all when strict is True... I will fix this later tonight. 😞