Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 Wait on vAppConfig during async create #817

Merged

Conversation

akutz
Copy link
Collaborator

@akutz akutz commented Dec 9, 2024

What does this PR do, and why is it needed?

This patch updates the behavior of the vAppConfig, LinuxPrep, and Sysprep bootstrap providers to wait on the VM to have a non-nil config.vAppConfig property when it is expected to exist.

Async create means we are no longer falling through to the update logic directly after create, instead, relying on the reconciler to be reentrant. That all works fine, but the problem is vpxd is sending signal on the property collector that the VM is ready prior to its config.vAppConfig property being set.

VM Op receives this signal and proceeds to reconcile the VM. Since previously we did not wait on an expected vAppConfig, we proceeded to configure and power on the VM. Once it is powered on, any customization that dependended upon the vAppConfig properties being set prior to first boot will no longer work.

This patch returns an error from GetOVFVAppConfigForConfigSpec if the VM has a nil config.vAppConfig.

Which issue(s) is/are addressed by this PR? (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):

Fixes NA

Are there any special notes for your reviewer:

The following steps were taken to verify the fix:

  1. Replace VM Op on Supervisor with an image built from this change.
  2. Apply the following VM:
    apiVersion: vmoperator.vmware.com/v1alpha3
    kind: VirtualMachine
    metadata:
      name: my-vm-1
      namespace: my-namespace-1
    spec:
      className: best-effort-small
      imageName: vmi-5119c3e3493b3cfe9 # ubuntu-server 24.10
      storageClass: wcpglobal-storage-profile
      bootstrap:
        linuxPrep: {}
        vAppConfig:
          properties:
          - key: password
            value:
              value: 'And another one bites the dust.'
  3. Look for key indicators from the VM Op log:
    1. Non-blocking create:
      I1209 16:01:56.384947       1 vmprovider_vm.go:208] "Doing a non-blocking create" logger="vsphere" vmName="my-namespace-1/my-vm-1"
    2. The reconciler backing off due to the vAppConfig not being available yet on the newly created VM:
      E1209 16:02:04.715968       1 virtualmachine_controller.go:323] "Failed to reconcile VirtualMachine" err="updating state failed with failed to create bootstrap data: vAppConfig is not yet available" logger="VirtualMachine" name="my-namespace-1/my-vm-1"
      I1209 16:02:04.721253       1 recorder.go:104] "updating state failed with failed to create bootstrap data: vAppConfig is not yet available" logger="events" type="Warning" object={"kind":"VirtualMachine","namespace":"my-namespace-1","name":"my-vm-1","uid":"7289b06a-11e7-44de-b2b3-fa3aed4c34df","apiVersion":"vmoperator.vmware.com/v1alpha3","resourceVersion":"3060028"} reason="UpdateFailure"
      
    3. The VM being created successfully with an IP and its vAppConfig property set as expected:
      1. It has an IP address:
        image
      2. It has the expected vAppConfig property:
        image

Please add a release note if necessary:

The VM controller now enter exponential backoff with an error until a VM that is expected to have a vAppConfig does so

This patch updates the behavior of the vAppConfig, LinuxPrep, and
Sysprep bootstrap providers to wait on the VM to have a non-nil
config.vAppConfig property when it is expected to exist.

Async create means we are no longer falling through to the update
logic directly after create, instead, relying on the reconciler to
be reentrant. That all works fine, but the problem is vpxd is sending
signal on the property collector that the VM is ready prior to its
config.vAppConfig property being set.

VM Op receives this signal and proceeds to reconcile the VM. Since
previously we did not wait on an expected vAppConfig, we proceeded
to configure and power on the VM. Once it is powered on, any customization
that dependended upon the vAppConfig properties being set prior to
first boot will no longer work.

This patch returns an error from GetOVFVAppConfigForConfigSpec if the
VM has a nil config.vAppConfig.
@github-actions github-actions bot added the size/M Denotes a PR that changes 30-99 lines. label Dec 9, 2024
@akutz akutz requested review from bryanv and sreyasn December 9, 2024 16:11
Copy link
Contributor

@dougm dougm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link

github-actions bot commented Dec 9, 2024

Code Coverage

Package Line Rate Health
github.com/vmware-tanzu/vm-operator/controllers/contentlibrary/clustercontentlibraryitem 82%
github.com/vmware-tanzu/vm-operator/controllers/contentlibrary/contentlibraryitem 86%
github.com/vmware-tanzu/vm-operator/controllers/contentlibrary/utils 97%
github.com/vmware-tanzu/vm-operator/controllers/infra/capability/configmap 86%
github.com/vmware-tanzu/vm-operator/controllers/infra/capability/crd 93%
github.com/vmware-tanzu/vm-operator/controllers/infra/configmap 71%
github.com/vmware-tanzu/vm-operator/controllers/infra/node 77%
github.com/vmware-tanzu/vm-operator/controllers/infra/secret 77%
github.com/vmware-tanzu/vm-operator/controllers/infra/validatingwebhookconfiguration 85%
github.com/vmware-tanzu/vm-operator/controllers/infra/zone 76%
github.com/vmware-tanzu/vm-operator/controllers/storageclass 95%
github.com/vmware-tanzu/vm-operator/controllers/storagepolicyquota 97%
github.com/vmware-tanzu/vm-operator/controllers/util/encoding 73%
github.com/vmware-tanzu/vm-operator/controllers/virtualmachine/storagepolicyusage 99%
github.com/vmware-tanzu/vm-operator/controllers/virtualmachine/virtualmachine 87%
github.com/vmware-tanzu/vm-operator/controllers/virtualmachine/volume 87%
github.com/vmware-tanzu/vm-operator/controllers/virtualmachineclass 75%
github.com/vmware-tanzu/vm-operator/controllers/virtualmachinepublishrequest 81%
github.com/vmware-tanzu/vm-operator/controllers/virtualmachinereplicaset 68%
github.com/vmware-tanzu/vm-operator/controllers/virtualmachineservice 83%
github.com/vmware-tanzu/vm-operator/controllers/virtualmachineservice/providers 92%
github.com/vmware-tanzu/vm-operator/controllers/virtualmachinesetresourcepolicy 80%
github.com/vmware-tanzu/vm-operator/controllers/virtualmachinewebconsolerequest/v1alpha1 72%
github.com/vmware-tanzu/vm-operator/controllers/virtualmachinewebconsolerequest/v1alpha1/conditions 88%
github.com/vmware-tanzu/vm-operator/controllers/virtualmachinewebconsolerequest/v1alpha1/patch 78%
github.com/vmware-tanzu/vm-operator/controllers/virtualmachinewebconsolerequest/v1alpha2 72%
github.com/vmware-tanzu/vm-operator/pkg/bitmask 100%
github.com/vmware-tanzu/vm-operator/pkg/builder 95%
github.com/vmware-tanzu/vm-operator/pkg/conditions 88%
github.com/vmware-tanzu/vm-operator/pkg/config 100%
github.com/vmware-tanzu/vm-operator/pkg/config/capabilities 100%
github.com/vmware-tanzu/vm-operator/pkg/config/env 100%
github.com/vmware-tanzu/vm-operator/pkg/context/generic 100%
github.com/vmware-tanzu/vm-operator/pkg/context/operation 100%
github.com/vmware-tanzu/vm-operator/pkg/patch 78%
github.com/vmware-tanzu/vm-operator/pkg/prober 91%
github.com/vmware-tanzu/vm-operator/pkg/prober/probe 90%
github.com/vmware-tanzu/vm-operator/pkg/prober/worker 77%
github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere 75%
github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/client 80%
github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/clustermodules 71%
github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/config 89%
github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/contentlibrary 74%
github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/credentials 100%
github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/network 80%
github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/placement 79%
github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/session 71%
github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/storage 44%
github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/sysprep 100%
github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/vcenter 82%
github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/virtualmachine 84%
github.com/vmware-tanzu/vm-operator/pkg/providers/vsphere/vmlifecycle 69%
github.com/vmware-tanzu/vm-operator/pkg/record 87%
github.com/vmware-tanzu/vm-operator/pkg/topology 91%
github.com/vmware-tanzu/vm-operator/pkg/util 87%
github.com/vmware-tanzu/vm-operator/pkg/util/annotations 100%
github.com/vmware-tanzu/vm-operator/pkg/util/cloudinit 89%
github.com/vmware-tanzu/vm-operator/pkg/util/cloudinit/validate 91%
github.com/vmware-tanzu/vm-operator/pkg/util/image 100%
github.com/vmware-tanzu/vm-operator/pkg/util/kube 89%
github.com/vmware-tanzu/vm-operator/pkg/util/kube/cource 100%
github.com/vmware-tanzu/vm-operator/pkg/util/kube/internal 100%
github.com/vmware-tanzu/vm-operator/pkg/util/kube/proxyaddr 75%
github.com/vmware-tanzu/vm-operator/pkg/util/kube/spq 100%
github.com/vmware-tanzu/vm-operator/pkg/util/netplan 100%
github.com/vmware-tanzu/vm-operator/pkg/util/ovfcache 75%
github.com/vmware-tanzu/vm-operator/pkg/util/ovfcache/internal 100%
github.com/vmware-tanzu/vm-operator/pkg/util/paused 100%
github.com/vmware-tanzu/vm-operator/pkg/util/ptr 100%
github.com/vmware-tanzu/vm-operator/pkg/util/resize 97%
github.com/vmware-tanzu/vm-operator/pkg/util/vmopv1 93%
github.com/vmware-tanzu/vm-operator/pkg/util/vsphere/client 64%
github.com/vmware-tanzu/vm-operator/pkg/util/vsphere/vm 79%
github.com/vmware-tanzu/vm-operator/pkg/util/vsphere/watcher 87%
github.com/vmware-tanzu/vm-operator/pkg/vmconfig 95%
github.com/vmware-tanzu/vm-operator/pkg/vmconfig/crypto 98%
github.com/vmware-tanzu/vm-operator/pkg/webconsolevalidation 100%
github.com/vmware-tanzu/vm-operator/services/vm-watcher 92%
github.com/vmware-tanzu/vm-operator/webhooks/common 100%
github.com/vmware-tanzu/vm-operator/webhooks/persistentvolumeclaim/validation 95%
github.com/vmware-tanzu/vm-operator/webhooks/unifiedstoragequota/validation 89%
github.com/vmware-tanzu/vm-operator/webhooks/virtualmachine/mutation 87%
github.com/vmware-tanzu/vm-operator/webhooks/virtualmachine/validation 95%
github.com/vmware-tanzu/vm-operator/webhooks/virtualmachineclass/mutation 62%
github.com/vmware-tanzu/vm-operator/webhooks/virtualmachineclass/validation 89%
github.com/vmware-tanzu/vm-operator/webhooks/virtualmachinepublishrequest/validation 92%
github.com/vmware-tanzu/vm-operator/webhooks/virtualmachinereplicaset/validation 90%
github.com/vmware-tanzu/vm-operator/webhooks/virtualmachineservice/mutation 67%
github.com/vmware-tanzu/vm-operator/webhooks/virtualmachineservice/validation 92%
github.com/vmware-tanzu/vm-operator/webhooks/virtualmachinesetresourcepolicy/validation 89%
github.com/vmware-tanzu/vm-operator/webhooks/virtualmachinewebconsolerequest/v1alpha1/validation 92%
github.com/vmware-tanzu/vm-operator/webhooks/virtualmachinewebconsolerequest/v1alpha2/validation 92%
Summary 83% (10685 / 12818)

Minimum allowed line rate is 79%

@akutz akutz merged commit f415352 into vmware-tanzu:main Dec 9, 2024
10 checks passed
@akutz akutz deleted the fix/wait-on-config.vAppConfig-when-async branch December 9, 2024 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-not-required size/M Denotes a PR that changes 30-99 lines.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants