Evaluate how we configure docker (and nodes in general) for Kubernetes

[colemickens]

This is regarding the scramble we had two weeks ago for the docker upgrade breaking clusters.

See the upstream issue: kubernetes/kubernetes#40182 which features suggestions beyond pinning to 1.12.x. Additionally, it's suggested that we simply tell docker to not configure iptables at all.

This partially serves as a reminder to see if sig-node has a list of published guidelines. If not, pinging thockin after he's back from vacation to see if he's aware of a list.

Also, as a reminder to watch upstream to see how they handle it (a patch in Kube, a recommendation to change docker's config json, etc).

[colemickens]

We should look at the doc that @brendandburns sent around from Deis when we do this.

[colemickens]

Also, we need to keep this under consideration for 1.6 and newer addon manager, etc:

• Supports 'ensure exist' class addon in Addon-manager #40108 - changes how addon manager treats annotations
• Create a default StorageClass for Azure - StorageClass addon with addon-manager will replace the existing admission controller.

[colemickens]

Also:
kubelet.go:1145] Image garbage collection failed: unable to find data for container /.

[anhowe]

closing as not specific enough.