Create conformance test for scs-0214-v1

[cah-hbaum]

As discussed in #481, we need to create a conformance test for SCS-0214-v1. This PR does that and also includes a separate commit containing the necessary changes for the Conformance Tests section of the standard. If we don't want to include this, the commit will be split up into a separate PR.

Resolves SovereignCloudStack/issues#477

[mbuechse]

A few things that I think should be addressed. If you are time-constrained, I offer to assist.

[mbuechse]

This seems kinda redundant given that the same info is in the yaml file?

[cah-hbaum]

This is there if a yaml file isn't provided.

[mbuechse]

I know. Then it should just be provided (and the fallback can be way more minimal)

[cah-hbaum]

Its already more minimal than the config, but maybe it can be minimized more(?) Not sure though.
I fixed some small artifacts I found in the config.

[martinmo]

Why should the user be able to configure the logging in the first place? I would actually remove the -c/--config option and the yaml file completely.

[mbuechse]

Let's come back to this point later. It's not mission critical.

[cah-hbaum]

Added a fixup containing the mentioned changes. I hope I didn't forget anything @mbuechse

Hint: I couldn't test this right now, I got some problems with my K8s clusters.

[martinmo]

Good work, thanks Hannes. But for me, some of the code was hard to understand/follow and I left some suggestions for improvement.

Additionally, with manual testing I could only verify that the following cases have the expected conformance test outcome:

• test with a minikube which is too small ("ERROR: Node is not labelled as a control plane node" and exit code 2)
• test with a k8s cluster with only one region ("WARNING: There seems to be no distribution across multiple regions or labels aren't set correctly across nodes." and exit code 0, i.e., conforming)

It would be nice if you could provide some test examples. If you reorganize the code in a way that the labels-by-node list can come from another source and not only from actually calling get_k8s_cluster_labelled_nodes(…) this would be easier to review.

[martinmo]

Why should the user be able to configure the logging in the first place? I would actually remove the -c/--config option and the yaml file completely.

[martinmo]

Additionally, with manual testing I could only verify that the following cases have the expected conformance test outcome

I forgot to mention that, to me, the testing logic seems to be ok. Still, it would be good to have an easy way to verify the different scenarios, without having access to different kinds of clusters.

[cah-hbaum]

Additionally, with manual testing I could only verify that the following cases have the expected conformance test outcome

I forgot to mention that, to me, the testing logic seems to be ok. Still, it would be good to have an easy way to verify the different scenarios, without having access to different kinds of clusters.

I agree with you on this, we can probably look into this later on. (Sadly not until tomorrow).

I also fixed/updated/changed most of the things you mentioned in the other comments, so you can recheck tomorrow =)

Edit: Also in response to #486 (comment), in a previous test, we wanted to provide a possibility to configure the logging however it is necessary. You could now use the same config from that test here.

[martinmo]

LGTM & we agreed via chat to add tests for the tests later.