Skip to content

kustomise doesn't work for admission controller webhook in preprogrammed mode #361

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
bart0sh opened this issue Apr 9, 2020 · 6 comments · Fixed by #362
Closed

kustomise doesn't work for admission controller webhook in preprogrammed mode #361

bart0sh opened this issue Apr 9, 2020 · 6 comments · Fixed by #362
Assignees
@askervin

Comments

@bart0sh
Copy link
Member

bart0sh commented Apr 9, 2020 •
edited
Loading

Steps to reproduce:

  1. Deploy webhook in preprogrammed mode:
> ./scripts/fpga-plugin-prepare-for-kustomization.sh 
Creating certs in /tmp/tmp.2rMAiJ4N0k
certificatesigningrequest.certificates.k8s.io/intel-fpga-webhook-svc.default created
NAME                             AGE   REQUESTOR          CONDITION
intel-fpga-webhook-svc.default   0s    kubernetes-admin   Pending
certificatesigningrequest.certificates.k8s.io/intel-fpga-webhook-svc.default approved
Removing /tmp/tmp.2rMAiJ4N0k

created for kustomization:
- /srv/demo/go/src/github.com/intel/intel-device-plugins-for-kubernetes/deployments/fpga_admissionwebhook/base/intel-fpga-webhook-certs-secret

> kubectl create -n default -k ./deployments/fpga_admissionwebhook/overlays/preprogrammed/
customresourcedefinition.apiextensions.k8s.io/acceleratorfunctions.fpga.intel.com created
mutatingwebhookconfiguration.admissionregistration.k8s.io/fpga-mutator-webhook-cfg created
clusterrole.rbac.authorization.k8s.io/fpga-reader created
clusterrolebinding.rbac.authorization.k8s.io/default-fpga-reader created
secret/intel-fpga-webhook-certs created
service/intel-fpga-webhook-svc created
deployment.apps/intel-fpga-webhook-deployment created
acceleratorfunction.fpga.intel.com/arria10.dcp1.0-compress created
acceleratorfunction.fpga.intel.com/arria10.dcp1.0-nlb0 created
acceleratorfunction.fpga.intel.com/arria10.dcp1.0-nlb3 created
acceleratorfunction.fpga.intel.com/arria10.dcp1.1-nlb0 created
acceleratorfunction.fpga.intel.com/arria10.dcp1.1-nlb3 created
acceleratorfunction.fpga.intel.com/arria10.dcp1.2-nlb0 created
acceleratorfunction.fpga.intel.com/arria10.dcp1.2-nlb3 created
acceleratorfunction.fpga.intel.com/d5005-nlb0 created
acceleratorfunction.fpga.intel.com/d5005-nlb3 created
  1. check its logs:
> kubectl logs intel-fpga-webhook-deployment-959c69d58-vhvbx
I0409 09:36:47.062536       1 controller.go:91] Starting controller
E0409 09:36:47.076528       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:36:48.078046       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:36:49.082048       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:36:50.084000       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:36:51.085891       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:36:52.087610       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:36:53.089483       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:36:54.091230       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:36:55.092983       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:36:56.094589       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:36:57.096734       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:36:58.098461       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:36:59.100231       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:37:00.101634       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:37:01.103238       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:37:02.104764       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:37:03.106426       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:37:04.108068       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:37:05.109567       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
E0409 09:37:06.111237       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)
  1. Try to create test pod:
> cat <<END | kubectl create -f-
apiVersion: v1
kind: Pod
metadata:
  name: test-fpga-af
spec:
  containers:
  - name: test-fpga-af
    image: intel/opae-nlb-demo:devel
    imagePullPolicy: IfNotPresent
    command: ['sleep', '10000']
    securityContext:
      capabilities:
        add:
          [IPC_LOCK]
    resources:
      limits:
        fpga.intel.com/arria10.dcp1.0-nlb0: 1
        cpu: 1
        hugepages-2Mi: 20Mi
  restartPolicy: Never
END
Error from server: error when creating "STDIN": admission webhook "fpga.mutator.webhooks.intel.com" denied the request: Unknown FPGA resource: fpga.intel.com/arria10.dcp1.0-nlb0
@bart0sh
Copy link
Member Author

bart0sh commented Apr 9, 2020

From the first look it seems that region CRDs are missed in preprogrammed mode. I'll try to add them and test.

@bart0sh
Copy link
Member Author

bart0sh commented Apr 9, 2020

@rojkov do we need region CRDs in preprogrammed mode?

@rojkov
Copy link
Contributor

rojkov commented Apr 9, 2020

No, for the preprogrammed mode they are not needed. And if #358 gets merged the region CRDs will be needed only to facilitate the case of regions programmed by workloads (see regiondevel mode of the FPGA plugin).

@bart0sh
Copy link
Member Author

bart0sh commented Apr 9, 2020

if it's not needed why the webhook complains that they're not found?

E0409 09:36:47.076528       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.FpgaRegion: the server could not find the requested resource (get fpgaregions.fpga.intel.com)

@rojkov
Copy link
Contributor

rojkov commented Apr 9, 2020

The built-in controller that watches CRDs doesn't respect mode and wants to monitor AFs and FpgaRegions irrespective to the current mode:

intel-device-plugins-for-kubernetes/cmd/fpga_admissionwebhook/controller.go

Line 78 in d4b366e

regionInformer.Informer().AddEventHandler(createEventHandler("region", controller.queue))
.

@bart0sh
Copy link
Member Author

bart0sh commented Apr 9, 2020

Seems to work after adding region CRDs to the base kustomise yamls. I'll create PR for that.

bart0sh added a commit to bart0sh/intel-device-plugins-for-kubernetes that referenced this issue Apr 9, 2020
@bart0sh
fpga webhook: fix deployment issue
7d8a33c 
Webhook uses region CRDs even if run in preprogrammed mode.

Adding them to the base configuration should fix this deployment error:
  Failed to list *v1.FpgaRegion: the server could not find the requested resource

Fixes: intel#361
@bart0sh bart0sh mentioned this issue Apr 9, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@askervin askervin

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fpga webhook: fix deployment issue bart0sh/intel-device-plugins-for-kubernetes
3 participants
@rojkov @askervin @bart0sh