Add a new results format: manual

[johnSchnake]

What this PR does / why we need it:
Manual results allow a user to write the sonobuoy_results.yaml
file largely on their own.

Due to some limitations we still wrap their file with one more
summary level item.

When the user gave us a single top-level item, the status from that
item is used for the summary. Otherwise all the top level items
are considered and a non-judgemental routine just counts the various
status values and shows them as a human readable status for the
top level. This helps ensure that the status value from custom output
can be various pieces of info the user may want to provide and not just
something like 'pass' or 'fail'.

Release note:

Sonobuoy now supports plugins providing their own results using the Sonobuoy results format. These results will be post-processed by Sonobuoy for inclusion in the `results` workflow. Sonobuoy will post-process any file listed in a plugin configuration under `result-files` or, if not provided, a `sonobuoy_results.yaml` file.

[johnSchnake]

not junit; just left over from copy. Update this.

[johnSchnake]

@zubron What do you think about this solution?

If we do something like this then we can summarize any custom status the user puts on the items. It seems like the only other alternative is to force them into a very narrow set of allowed values (passed/failed/skipped) which seemed to me to miss the point of allowing completely custom results from the plugin.

[zubron]

I think this is a reasonable approach. You make a good point about the narrow set of allowed values, I think it will be preferable to allow custom statuses there 👍

[johnSchnake]

Will need some good tests for this; I have some but I think it needs changes/tests to cover the daemonset case.

Not only will the top-summary need its status, but the objects right below the top level summary are the 'node summary' so we need to make sure that we arent leaving one layer blank of status values.

[zubron]

I've added tests to this and made some changes to the aggregation of results and which files Sonobuoy will look for during results processing.

Now, Sonoboy will only process files explicitly listed in the plugin results files, or any file named sonobuoy_results.yaml. Given that we are expecting files of a particular format, I thought it would be best to process only those files which we expect to be in the Sonobuoy results format. Users can still add files of other formats to their results tarballs but sonobuoy will only post process those listed in the result files or sonobuoy_results.yaml. Within the sonobuoy_results.yaml file produced by Sonobuoy, any name provided by the user in their results files is discarded in favour of using the filename from where the results were retrieved. The root node name will match the plugin name so it will exist in the post processed results file. This removes the need for users to have to write a matching name in their results file.

For the overall status aggregation, how the results are aggregated depends on whether the plugin is a Job or DaemonSet plugin.

For Job plugins, if only one results file is processed, the status in that file is used as the overall status for plugin. If multiple files are processed, then the statuses are grouped by count, and the overall status for the plugin is a human readable form of that grouping, e.g. custom-status: 1, another-custom-status: 2.

For DaemonSet plugins, each node's status is calculated as for a Job plugin to handle the case where a DS plugin has multiple result files. However, the overall plugin status is determined based on each item from each node rather than the overall status from each node. This avoids already aggregated statuses being used within the overall aggregate status. If the same status is used for all files for all nodes then that will be the overall status as in the Job case.
For example, for DS plugin with 3 nodes:

node1:
 - item1status: success
 - item2status: success
node2:
 - item1status: error
 - item2status: success
node3:
 - item1status: success
 - item2status: failure

This will result in the following statuses:

// Overall plugin status, calculated from each nodes items, grouped by count
overall: success: 4, failure: 1, error: 1
// Node1 status, all the same so that is used
node1: success
// Node2 and 3, grouped by count
node2: success: 1, error: 1
node3: success: 1, failure: 1

The documentation will need to be updated to include a guide on how to write custom results but this will be done in a follow up PR.