fake client unable to List Job objects

[raffaelespazzoli]

I am getting this error when trying to list job objects using the fake client in my unit tests:

{"level":"error","ts":1563402934.2386599,"logger":"controller_gitopsconfig","msg":"error retrieving the job","error":"item[0]: can't assign or convert unstructured.Unstructured into v1.Job","stacktrace":"github.com/KohlsTechnology/eunomia/vendor/github.com/go-logr/zapr.(*zapLogger).Error\n\t/home/rspazzol/go/src/github.com/KohlsTechnology/eunomia/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/KohlsTechnology/eunomia/test/unit.TestDeleteRemovingFinalizer\n\t/home/rspazzol/go/src/github.com/KohlsTechnology/eunomia/test/unit/gitopsconfig_controller_test.go:372\ntesting.tRunner\n\t/usr/lib/golang/src/testing/testing.go:865"}

the code to reproduce the error is the following:

	// Create the deleteJob
	err = cl.Create(context.TODO(), deleteJob)
	if err != nil {
		log.Error(err, "Create Job - Failed creating Job type action of Delete")
	}

	jobList := &batchv1.JobList{}
	err = cl.List(context.TODO(), &client.ListOptions{
		//Namespace: nsn.Namespace,
	}, jobList)
	if err != nil {
		log.Error(err, "error retrieving the job")
	}

here is some of the initialization that may be relevant:

deleteJob = &batchv1.Job{
		TypeMeta: metav1.TypeMeta{
			Kind:       "Job",
			APIVersion: "batch/v1",
		},
		ObjectMeta: metav1.ObjectMeta{
			Name:      deleteJobName,
			Namespace: namespace,
			Labels:    map[string]string{"action": "delete"},
			OwnerReferences: []metav1.OwnerReference{
				{
					APIVersion:         "eunomia.kohls.io/v1alpha1",
					Kind:               "GitOpsConfig",
					Name:               name,
					Controller:         &controller,
					BlockOwnerDeletion: &blockDelete,
				},
			},
		},
		Spec: batchv1.JobSpec{
			Parallelism:  &parallelism,
			Completions:  &completions,
			BackoffLimit: &backoffLimit,
		},
		Status: batchv1.JobStatus{
			Succeeded: 2,
		},
	}

	s := scheme.Scheme
	apis.AddToScheme(s)
	// Initialize fake client
	cl := fake.NewFakeClient(objs...)

[raffaelespazzoli]

after adding some log statements to the fake client code I can see that this statement in fixture.go is successful and I can see that the object in the array is actually a Job:

object objs: %+v [0xc000011740 &Job{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:gitops-operator-delete,GenerateName:,Namespace:gitops,SelfLink:,UID:,ResourceVersion:,Generation:0,CreationTimestamp:0001-01-01 00:00:00 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{action: delete,},Annotations:map[string]string{},OwnerReferences:[{eunomia.kohls.io/v1alpha1 GitOpsConfig gitops-operator  0xc000885114 0xc000885115}],Finalizers:[],ClusterName:,Initializers:nil,},Spec:JobSpec{Parallelism:*1,Completions:*1,ActiveDeadlineSeconds:nil,Selector:nil,ManualSelector:nil,Template:k8s_io_api_core_v1.PodTemplateSpec{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:,GenerateName:,Namespace:,SelfLink:,UID:,ResourceVersion:,Generation:0,CreationTimestamp:0001-01-01 00:00:00 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{},Annotations:map[string]string{},OwnerReferences:[],Finalizers:[],ClusterName:,Initializers:nil,},Spec:PodSpec{Volumes:[],Containers:[],RestartPolicy:,TerminationGracePeriodSeconds:nil,ActiveDeadlineSeconds:nil,DNSPolicy:,NodeSelector:map[string]string{},ServiceAccountName:,DeprecatedServiceAccount:,NodeName:,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:nil,ImagePullSecrets:[],Hostname:,Subdomain:,Affinity:nil,SchedulerName:,InitContainers:[],AutomountServiceAccountToken:nil,Tolerations:[],HostAliases:[],PriorityClassName:,Priority:nil,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[],RuntimeClassName:nil,EnableServiceLinks:nil,},},BackoffLimit:*1,TTLSecondsAfterFinished:nil,},Status:JobStatus{Conditions:[],StartTime:<nil>,CompletionTime:<nil>,Active:0,Succeeded:2,Failed:0,},}]

same this at this statement.

object matchingObjs: %+v [0xc000011740 &Job{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:gitops-operator-delete,GenerateName:,Namespace:gitops,SelfLink:,UID:,ResourceVersion:,Generation:0,CreationTimestamp:0001-01-01 00:00:00 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{action: delete,},Annotations:map[string]string{},OwnerReferences:[{eunomia.kohls.io/v1alpha1 GitOpsConfig gitops-operator  0xc000885114 0xc000885115}],Finalizers:[],ClusterName:,Initializers:nil,},Spec:JobSpec{Parallelism:*1,Completions:*1,ActiveDeadlineSeconds:nil,Selector:nil,ManualSelector:nil,Template:k8s_io_api_core_v1.PodTemplateSpec{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:,GenerateName:,Namespace:,SelfLink:,UID:,ResourceVersion:,Generation:0,CreationTimestamp:0001-01-01 00:00:00 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{},Annotations:map[string]string{},OwnerReferences:[],Finalizers:[],ClusterName:,Initializers:nil,},Spec:PodSpec{Volumes:[],Containers:[],RestartPolicy:,TerminationGracePeriodSeconds:nil,ActiveDeadlineSeconds:nil,DNSPolicy:,NodeSelector:map[string]string{},ServiceAccountName:,DeprecatedServiceAccount:,NodeName:,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:nil,ImagePullSecrets:[],Hostname:,Subdomain:,Affinity:nil,SchedulerName:,InitContainers:[],AutomountServiceAccountToken:nil,Tolerations:[],HostAliases:[],PriorityClassName:,Priority:nil,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[],RuntimeClassName:nil,EnableServiceLinks:nil,},},BackoffLimit:*1,TTLSecondsAfterFinished:nil,},Status:JobStatus{Conditions:[],StartTime:<nil>,CompletionTime:<nil>,Active:0,Succeeded:2,Failed:0,},}]

then at this line I get the above error. So it looks like the meta.SetList function fails.
Can someone help me understand why?

[xcoulon]

@raffaelespazzoli We just faced the same issue in a test, after applying objects from a template. In this test, we created a namespace provided by a template, but when calling the List method on the fake client, we would get the same error as you reported.
We found a workaround by converting the runtime.Object into a corev1.Namespace (using scheme.Convert(obj, ns, nil) before passing it to the fake client, so its underlying object tracker would have an object of type corev1.Namespace which would be compatible with the dest type when calling the client.List(context, opts, namespaceList) method.

Hope this helps, although granted, it's a workaround on the client side, and we knew we were dealing with Namespace kind of objects, so that was pretty straightforward in our case.

[jessehu]

Hi @xcoulon could you share the sample code of your workaround please ? I hit the same issue.

[DirectXMan12]

/kind bug
/priority backlog

[DirectXMan12]

/area fake-client

[vincepri]

/priority awaiting-more-evidence
/help

[k8s-ci-robot]

@vincepri:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/priority awaiting-more-evidence
/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[vincepri]

/lifecycle frozen

[williamaronli]

hi meet the same issue @xcoulon could you share us how you deal with this issue?

can't assign or convert unstructured.Unstructured into v1alpha3.WorkloadGroup

[alvaroaleman]

I believe this got fixed in #1662 which is available in controller-runtime 0.10.1 onwards. Please feel free to comment or re-open if that is not the case.