Breaking API changes for TestReport in Jenkins 2

[tom-smalls]

Disclaimer: I've only noticed this in Jenkins version 2.7.3

I noticed I was getting 0 back for the TestReport.getTotalCount() yesterday after looking into the upgrade process from a Jenkins 1.6.X server to Jenkins 2. After looking at the JSON API for test report on Jenkins 2, I noticed that there is no totalCount attribute anymore, with the following available:

"empty" : false,
"failCount" : 0,
"passCount" : 458,
"skipCount" : 2,

I'm not sure what the best approach would be to "fix" this whilst maintaining backward compatibility. Potentially looking for (and summing) failCount, passCount, and skipCount if totalCount returns 0 in the library? As far as I'm aware, passCount is new as well. Alternatively, doing something with the empty flag.

What are your thoughts?

[khmarbaise]

First thanks for you hints about that but that's what I feared...that this would happened. I already have some information in the code which contains the version of Jenkins is running..So it looks I need either to go with those differences (I don't know how at the moment) or just focus on version 2.X of Jenkins?
Within the integration tests I'm trying to find a good way to handle different jenkins versions which is really hard at the moment...The way I tried does not really work...

[tom-smalls]

So something else I've been thinking about as a solution is to slightly
change your versioning and release cycle to be tightly coupled to the
Jenkins version ie 2.7.3 of the library supports Jenkins 2.7.3

It would resolve a lot of your issues around backwards compatibility. The
breakages and issues that I suspect occur with the library are usually down
to newer versions of Jenkins having breaking changes to the api. This would
also remove the need for hacks to support backward compatibility

On Sunday, 4 September 2016, Karl Heinz Marbaise notifications@github.com
wrote:

First thanks for you hints about that but that's what I feared...that this
would happened. I already have some information in the code which contains
the version of Jenkins is running..So it looks I need either to go with
those differences (I don't know how at the moment) or just focus on version
2.X of Jenkins?
Within the integration tests I'm trying to find a good way to handle
different jenkins versions which is really hard at the moment...The way I
tried does not really work...

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#184 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGydjbBO4wCx_lFrJb44dtvoG-8WLUqbks5qmp6ZgaJpZM4J0e6B
.

[tom-smalls]

The advantage of this for me is when Jenkins make an API breaking change between versions I'll get a compile error in the corresponding library version, rather than finding out by accident something isn't quite working!

[tom-smalls]

I just realised this isn't a 2.X breaking change as it also appears to be the case for 1.658

[khmarbaise]

After thinking and testing about this I realized there is something wrong here...You have mentioned you have used TestReport.getTotalCount() within TestReport class which has been introduced into 0.3.1 of the API...Furthermore I have tested with 2.19.2, 2.7.1, 1.651.3 where I can't see any difference in the REST API of Jenkins (related to totalCount etc.). But what I have found is that you are referencing the TestResult (based on "empty" : false,) which is part of TestResult class which also contains the "empty" entry.

        JenkinsServer js = new JenkinsServer(URI.create("http://localhost:10090/buildserver"), "admin", "admin");
        
        JobWithDetails job = js.getJob("maven-test");
        Build lastCompletedBuild = job.getLastCompletedBuild();
        TestReport testReport = lastCompletedBuild.getTestReport();
        
        System.out.println("Fail Count:" + testReport.getFailCount());
        System.out.println("Skip Count:" + testReport.getSkipCount());
        System.out.println("Total Count:" + testReport.getTotalCount());
        List<TestChildReport> childReports = testReport.getChildReports();
        
        for (TestChildReport testChildReport : childReports) {
            System.out.println("Child: ");
            TestResult result = testChildReport.getResult();
            System.out.println("  Duration: " + result.getDuration());
            System.out.println("  Fail Count:" + result.getFailCount());
            System.out.println("  Pass Count:" + result.getPassCount());
            System.out.println("  Skip Count:" + result.getSkipCount());
            
            List<TestSuites> suites = result.getSuites();
            System.out.println("Suites:");
            
            for (TestSuites testSuites : suites) {
                List<TestCase> cases = testSuites.getCases();
                for (TestCase testCase : cases) {
                    
                }
            }
        }

May I misunderstand a thing here...but at the moment I don't see an issue...May be you can clear that up....

[khmarbaise]

So after more testing I found the misunderstanding on my site. I assume you have tried to get TestReport of a unit test of a build which is NOT a Maven build..? There is a difference between TestReport which is coming from a Maven build and a freestyle or non Maven build...which seemed to be the issue...So i will investigate that more...

[tom-smalls]

Thanks, I'll test this out when I have a chance (either today or tomorrow)

[khmarbaise]

Cool. This is great if you check this...Take your time...

[tom-smalls]

This works great, thanks!