Use multiple formatters in the same test run

[lonnblad]

Is your feature request related to a problem? Please describe.
When running the JUnit and Cucumber formatters, I don't get any feedback.

Describe the solution you'd like
To be able to use multiple formatters so that I can run the Progress formatter next to JUnit/Cucumber.

Describe alternatives you've considered

Additional context

[jesusnoseq]

Hi,
Is there anyone working on this feature?
I would like to propose a solution

[lonnblad]

Hi @jesusnoseq, I don't think anyone is working on it, so feel free! 😊

[lonnblad]

the idea was to have the CLI looking something like this:

godog run \
 --format progress \
 --format junit:report.xml \
 --concurrency 10 \
 features

[jesusnoseq]

Ok!
This what I plan to do

Modify runWithOptions function

• Parse command
  • --format junit:report.xml,cucumber:report.json,progress
• Refactor formatter finder and creation
• Handle output creation
• Validation foar multiple formatters
  • Disallow repetition of output.
  • Allow formatter repetition?
  • Any other validation to take into account?
• Handle errors file.close()
• Make printStepDefinitions always use os.Stdout
• Add test
  • Test_MultiFormatOutputRun and Test_MultiFormatOutputRunError
  • Parse format option with multiple formats

Create a new Formatter that wraps others

type MultiFormatter{
  fmts []Formatter
}

• Implement Formatter interface
• It will not be registered in the available formatter list
• Add simpler test than the other formatters

Add doc

• Example of how to launch godog with multiple output formats
• Update changelog with the changes made

My intention is to make as few changes as possible that is why I have come up with
--format junit:report.xml,cucumber:report.json
Instead of
--format junit:report.xml --format cucumber:report.json

[lonnblad]

I'm good with using comma separation and MultiFormatter sounds like a good idea!

My opinion is to allow formatter repition, even though I don't see a real use case for it, but I feel like we can leave that to be up to the user.

I also think that we should disallow repition of output and remember to check stdout there as well.

Thanks!!

[mdelapenya]

An use case could be on CI: I want to populate jUnit's XML format so that jenkins displays tests results AND cucumber's JSON format to also display scenarios with this rich UI: https://plugins.jenkins.io/cucumber-reports/

[lonnblad]

Hi @jesusnoseq, did you have a chance to look more at this issue? :)

[vearutop]

I've created a PR based on ideas discussed in this issue, please check if it is sufficient. It implements support for --format junit:report.xml,cucumber:report.json flag format.

With few more changes I think --format junit:report.xml --format cucumber:report.json can also be supported, please let me know if it is important.

[bahlo]

Awesome, can‘t wait to use this @vearutop!