500 response with CronJob in repo

[squaremo]

When we put a CronJob resource in our git repo, fluxd started returning 500s to release requests for another resource, citing only the server could not find the requested resource.

NB this was running Kubernetes 1.7, which doesn't support CronJob; it failed to sync that resource, while otherwise succeeding.

[squaremo]

That particular error message comes from the kubernetes client, grep tells me.

[squaremo]

On our dev (where we have the problem):

$ kubectl api-versions
apiextensions.k8s.io/v1beta1
apiregistration.k8s.io/v1beta1
apps/v1beta1
apps/v1beta2
authentication.k8s.io/v1
authentication.k8s.io/v1beta1
authorization.k8s.io/v1
authorization.k8s.io/v1beta1
autoscaling/v1
autoscaling/v2beta1
batch/v1
batch/v1beta1
certificates.k8s.io/v1beta1
extensions/v1beta1
networking.k8s.io/v1
policy/v1beta1
rbac.authorization.k8s.io/v1
rbac.authorization.k8s.io/v1beta1
storage.k8s.io/v1
storage.k8s.io/v1beta1
v1

and in minikube (where I can't reproduce the problem):

$ kubectl api-versions                                                                                                                                                                                             
admissionregistration.k8s.io/v1alpha1               
apiextensions.k8s.io/v1beta1                        
apiregistration.k8s.io/v1beta1                      
apps/v1beta1                                        
apps/v1beta2                                        
authentication.k8s.io/v1                            
authentication.k8s.io/v1beta1                       
authorization.k8s.io/v1                             
authorization.k8s.io/v1beta1                        
autoscaling/v1                                      
autoscaling/v2beta1                                 
batch/v1                                            
batch/v1beta1                                       
batch/v2alpha1                                      
certificates.k8s.io/v1beta1                         
extensions/v1beta1                                  
networking.k8s.io/v1                                
policy/v1beta1                                      
rbac.authorization.k8s.io/v1                        
rbac.authorization.k8s.io/v1alpha1                  
rbac.authorization.k8s.io/v1beta1                   
settings.k8s.io/v1alpha1                            
storage.k8s.io/v1                                   
storage.k8s.io/v1beta1                              
v1                      

The only possibly relevant difference is batch/v2alpha1, and I'm not sure how it's relevant, since the resource in question says it's v1alpha1.

[squaremo]

Except: https://github.com/weaveworks/flux/blob/master/cluster/kubernetes/resourcekinds.go#L225

/////////////////////////////////////////////////////////////////////////////
// batch/v2alpha1 CronJob

type cronJobKind struct{}

[rade]

There appear to be two distinct problems here:

1. flux or k8s disliking cronjobs in some circumstances
2. that dislike preventing releases of other resources

(2) is the more serious issue, by far - no problem with an individual resource should affect operations on others.

[squaremo]

no problem with an individual resource should affect operations on others.

Yes and no. For some operations we want to know whether we can correctly apply a change, and that usually means looking at everything to make sure there are no duplicates, or whatever.

In this case, I think there's a spot where we're over-estimating the information we need in order to proceed. It's OK to parse all the files to make sure there's a coherently-defined set of controllers (https://github.com/weaveworks/flux/blob/master/release/context.go#L68); less OK is asking for all the resources in question from kubernetes, then filtering down to the single one we care about (a few lines on).

[squaremo]

that dislike preventing releases of other resources

#869 addresses this problem, by making the release process only ask the cluster about services that are explicitly included in the release. It won't help if the problematic resource is specifically included, in implicitly included with --all.

[squaremo]

As for the specific problem of dealing with CronJobs: our code uses the batch/v2alpha1 extension to the client, and it seems likely that the API server in dev is just saying "I don't recognise that API extension".

I don't know how one is supposed to move between versions -- either using batch/v1alpha1 instead will just work, or we'll have to ask both APIs (and prepare for idk errors).

[squaremo]

As for the specific problem of dealing with CronJobs

This bit addressed by #875 (yes, it does respond with resources given either of batch/v2alpha1 or batch/v1beta1).