Feature/cucumber junit report

[janluak]

Hi pytest-bdd team,

thanks for the great work! Totally in love with BDD and awesome it works out of the box with pytest 🎉

At our project needed a junit formatted report for the output of the bdd tests (gitlab isn't providing a native support for other pytest-bdd reports).
Since I didn't like the idea of exporting the cucumberjson and then using a third party to convert to junit-xml I simply added the support natively to pytest-bdd with this PR.
Additionally, I fixed #567 (at least for steps, not a full feature) with it.

I am not really happy with the implementation of cucumber reports → if you approve the first version here I'd provide a rewrite of the reporting which is a bit cleaner and easier to adopt the additional feature list of cucumber reports. E.g. I'd not use the json format to generate the xml - afaik the different reports have different capabilities which doesn't help converting them from one to another when having the source data available...

Anyways, happy to get feedback ;)

[youtux]

This sounds like a good feature to have.

I wonder if we should better instead create the report using the Cucumber message format, and then use functionality already implemented in https://github.com/cucumber/junit-xml-formatter/ to convert from Cucumber Message format to JUnit (maybe we can improve that repo to provide a command line interface to convert from Cucumber Message to JUnit)

[jsa34]

I think using messages would help somewhat and we probably are better off investing in that route for longer term maintenance.

Having said that, the PR for messages for Python impl seems stuck: cucumber/messages#165

I wonder how to unblock/move forward...

[janluak]

Hi @youtux and @jsa34,

thanks for the feedback and the ref to messages.

In terms of using a separate package for changing the format I have two thoughts: one the one hand I definitely see the advantage of just being able to rely on a third party for the conversion. On the other hand this requires an additional installation of a different language which e.g. makes pipelines unnecessary big.

Yet, since messages seems to be the new standard in cucumber, (future) plugins will take care of making the generation of a separate report irrelevant for using them in CI.

With that in mind: do you want to temporarily follow up here and eventually replace the hole reporting as soon as messages come available or do you want to stress the messages implementation for an overall solution right away?