Version specific conformance test images

[jrnt30]

Currently the e2e plugin suggests that it is not advisable to run the E2E suite on clusters < 1.7+ due to the upstream issue of not having gates around the tests that are available. Instead of relying on that upstream change to be made and people "doing the right thing" all the time, would it be possible to simply have a multi-version build for the kube-conformance image, where each is tagged for the appropriate K8 version?

Selfishly I would like to use this on a few 1.6 clusters however it also seems like it may be a bit more stable.

The main deficiency I see is if there are newer tests created to validate additional aspects of the cluster that are compatible with lower environments, those changes wouldn't be included.

[bzub]

I just happen to have a PR open about this: heptio/kube-conformance#2

You can try out my changes, I'd be happy to hear how it works for you. You'd have to upload images to your own repo.

[timothysc]

We can/should do both.

We've already pushed gates upstream which detect version and api capability, but we should also version our kube-conformance container for multiple versions.

I'm ooto this week, but will update shortly.
/cc @kensimon

[timothysc]

This is linked with #17 so I'm going to close this one and try to test across revisions.

[timothysc]

@jrnt30 did you have a list of tests that you saw failing btw.