Support for doctest

[onqtam]

doctest 2.3 was released a few days ago and now supports XML output which is very similar to that of Catch2. What would it take to integrate it into report-ci?

[klemens-morgenstern]

I would love too do that, but it depends on the complexity, do you have a file that describes the schema? If I use the examples in the main project, does that cover it?

I am currently working on compiler log support, so it might take a few days until I get around to it, but if everything goes well I should be able to do it withing 3-4 weeks.

If you want, I would appreciate your input into the overview site and images for the check_run. Since you're the author you might have a few ideas how that should look. Though please note that it's markdown. I can render pictures on the backend, currently doing that using d3 on nodejs.

[onqtam]

To be honest I haven't used report.ci yet - but a friend suggested that it would be cool to have doctest supported, and recently @Milerius asked about it as well. Today I started a new job and I'm buried in other work so I won't be able to look at check_run...

About the XML format - it is a custom XML format just like that of Catch. Here (and here for subcases) is what doctest's XML output looks like - very similar to that of Catch - with a few tags renamed and 1 other difference - the Group tags in Catch are opened just once and closed at the end - basically not used. For doctest there are TestSuite tags and while executing tests if the current test is from a different TestSuite that the previous test - the TestSuite tag is closed and reopened with the new one.

At the end of the entire run there are these 2 tags which should be self explanatory:

• OverallResultsAsserts
• OverallResultsTestCases

I'll try to help with explaining anything on the doctest side.

[klemens-morgenstern]

Not a bad format. Also different enough from every other one, I can auto-detect it. It's in the backend now, will integrate it into the script soo. I could create a PR against your repo, but then it would always flag your build as "failing". Is that ok for you?

https://github.com/report-ci/doctest/runs/87187638

[Milerius]

It's so gorgeous omg

[klemens-morgenstern]

Alright, it works from travis, here's the result https://github.com/report-ci/doctest/runs/87455527

@onqtam , if you want to add it, here's my code https://github.com/report-ci/doctest/blob/travis/.travis.yml#L350

@Milerius , if you want to use it here's how that works (-s includes passing tests, so report.ci displays them too)

my_test --reporters=xml --out=my_test.xml
# Only looks for xml files!
curl https://report.ci/upload.py | python 

[onqtam]

@klemens-morgenstern thanks for the fast work - looks really great! I'll probably add it soon to the repo :)