Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VM always changed in eptRviMode,hvMode #472

Open
cultpony opened this issue Apr 12, 2024 · 11 comments
Open

VM always changed in eptRviMode,hvMode #472

cultpony opened this issue Apr 12, 2024 · 11 comments
Labels
bug/diff kind/bug related to Pulumi generating wrong diffs on preview or up. kind/bug Some behavior is incorrect or out of spec

Comments

@cultpony
Copy link

What happened?

I attempted to manage several VMs in a vSphere 8 installation via Pulumi, the VMs are created with cloud-init configuration and from a template, but eptRviMode and hvMode are not explicitly set at all.

Example

new vsphere.VirtualMachine("vm1", {
  name: "vm1",
  clone: { templateUuid: template.uuid },
  networkInterfaces: [ <omitted for brevity> ],
  disks: [ <omitted for brevity> ],
  <remainder omitted for brevity>,
})

Output of "pulumi preview" after completing a "pulumi up"

Previewing update (staging):
     Type                                     Name                   Plan       Info
     pulumi:pulumi:Stack                      <omitted>                   7 messages
     └─ vsphere:index:Folder                 <omitted>                       
 ~      └─ vsphere:index:VirtualMachine       <omitted>       update     [diff: ~eptRviMode,hvMode]
 ~         ├─ vsphere:index:VirtualMachine    <omitted>       update     [diff: ~eptRviMode,hvMode]
 ~         ├─ vsphere:index:VirtualMachine    <omitted>       update     [diff: ~eptRviMode,hvMode]
 +         └─ pulumi-nodejs:dynamic:Resource  <omitted>-kubeconfig  create     

Running "pulumi up" does not change this output but does cause the VMs to be shutdown, reconfigured and then started again.

Output of pulumi about

CLI          
Version      3.112.0
Go Version   go1.22.1
Go Compiler  gc

Plugins
NAME       VERSION
cloudinit  1.4.1
command    0.9.2
nodejs     unknown
random     4.16.0
vsphere    4.9.2

Host     
OS       darwin
Version  14.4.1
Arch     arm64

This project is written in nodejs: executable='/opt/homebrew/bin/node' version='v21.7.1'

Current Stack: <omitted>

TYPE                                         URN
pulumi:pulumi:Stack                          urn:pulumi:staging::vsphere::pulumi:pulumi:Stack::<omitted>
pulumi:providers:vsphere                     urn:pulumi:staging::vsphere::pulumi:providers:vsphere::default_4_9_2
pulumi:providers:random                      urn:pulumi:staging::vsphere::pulumi:providers:random::default_4_16_0
random:index/randomPassword:RandomPassword   urn:pulumi:staging::vsphere::random:index/randomPassword:RandomPassword::<omitted>-k8sSecretToken
vsphere:index/folder:Folder                  urn:pulumi:staging::vsphere::vsphere:index/folder:Folder::<omitted>
vsphere:index/virtualMachine:VirtualMachine  urn:pulumi:staging::vsphere::vsphere:index/folder:Folder$vsphere:index/virtualMachine:VirtualMachine::<omitted>
pulumi:providers:command                     urn:pulumi:staging::vsphere::pulumi:providers:command::default_0_9_2
command:local:Command                        urn:pulumi:staging::vsphere::vsphere:index/folder:Folder$vsphere:index/virtualMachine:VirtualMachine$command:local:Command::cat-k3s-kubeconf
pulumi:providers:pulumi-nodejs               urn:pulumi:staging::vsphere::pulumi:providers:pulumi-nodejs::default
vsphere:index/virtualMachine:VirtualMachine  urn:pulumi:staging::vsphere::vsphere:index/folder:Folder$vsphere:index/virtualMachine:VirtualMachine$vsphere:index/virtualMachine:VirtualMachine::<omitted>
vsphere:index/virtualMachine:VirtualMachine  urn:pulumi:staging::vsphere::vsphere:index/folder:Folder$vsphere:index/virtualMachine:VirtualMachine$vsphere:index/virtualMachine:VirtualMachine::<omitted>


OPP TYPE  URN
updating  urn:pulumi:staging::vsphere::vsphere:index/folder:Folder$vsphere:index/virtualMachine:VirtualMachine$vsphere:index/virtualMachine:VirtualMachine::<omitted>
updating  urn:pulumi:staging::vsphere::vsphere:index/folder:Folder$vsphere:index/virtualMachine:VirtualMachine$vsphere:index/virtualMachine:VirtualMachine::<omitted>


Backend        
Name           <omitted>
URL            file://~
User           <omitted>
Organizations  
Token type     personal

Dependencies:
NAME               VERSION
@infisical/sdk     2.1.8
@pulumi/command    0.9.2
@pulumi/random     4.16.0
@pulumi/vsphere    4.9.2
@types/node        18.19.21
crypto-js          4.2.0
js-yaml            4.1.0
@pulumi/cloudinit  1.4.1
@pulumi/pulumi     3.108.1
@types/crypto-js   4.2.2
@types/js-yaml     4.0.9

Pulumi locates its logs in /var/folders/_s/7t01k6l903q67w46sjhf5b3w0000gn/T/ by default

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

@cultpony cultpony added kind/bug Some behavior is incorrect or out of spec needs-triage Needs attention from the triage team labels Apr 12, 2024
@guineveresaenger guineveresaenger added bug/diff kind/bug related to Pulumi generating wrong diffs on preview or up. and removed needs-triage Needs attention from the triage team labels Apr 12, 2024
@guineveresaenger
Copy link
Contributor

Hi @cultpony - thank you for letting us know about this.

This may be an instance of pulumi/pulumi-terraform-bridge#1785, and may have a generic fix for that in the future.

@cultpony
Copy link
Author

I've also encountered a problem when trying to work around the issue; trying to ignore the properties causes an exception in the code somewhere, breaking the workflow entirely.

@nithom
Copy link

nithom commented Apr 18, 2024

Same problem here. But at least with the Python SDK and version 4.10.0 of the vSphere provider, setting pulumi.ResourceOptions(ignore_changes=["eptRviMode", "hvMode"]) for the VM seems to work fine without exceptions so far.

Edit: After doing a pulumi refresh, we run into exceptions too unfortunately, happens during preview:

error: vsphere:index/virtualMachine:VirtualMachine resource 'test-vm' has a problem: expected hv_mode to be one of ["hvAuto" "hvOn" "hvOff"], got . Examine values at 'test-vm.hvMode'.
error: vsphere:index/virtualMachine:VirtualMachine resource 'test-vm' has a problem: expected ept_rvi_mode to be one of ["automatic" "on" "off"], got . Examine values at 'test-vm.eptRviMode'.

Before the refresh, running up created the resource in the first run and displayed no changes for the second run. After refreshing the state, the property seems to go missing. Explicitly setting the two options on the VM resource doesn't help either.

@mikhailshilkov
Copy link
Member

@guineveresaenger @t0yv0 Can we rollout PlanResourceChange to this resource to see if that helps?

@t0yv0
Copy link
Member

t0yv0 commented May 10, 2024

Perhaps we could add an env var to quickly try this without rebuilding. That could help to quickly figure it out if there's a repro at hand.

@mikhailshilkov
Copy link
Member

@cultpony Could you please provide a larger self-sufficient repro without omitting most properties? That would help us troubleshoot the issue. Of course, replace any sensitive information.

@cultpony
Copy link
Author

A new property has begun also exhibiting this problem: syncTimeWithHost

I took the code I'm using and removed any outside references as much as possible, not interfering with any of the actual properties being set, you'll have to obviously spin up your own vsphere cluster and substitute the IDs involved. Removing "ignoreChanges: [ "syncTimeWithHost" ]" causes that to also always be changed. Ignoring changes on eptRviMode or hvMode causes the entire thing to break entirely (throws an exception).

let metadata = Buffer.from(`# cloud-init\n`, "binary")
    .toString("base64");

let userdata = Buffer.from(`#cloud-config\n`, "binary")
    .toString("base64");

let template = await vsphere.getVirtualMachine({
    name: "TEMPLATE_NAME",
    datacenterId: "DC_ID",
});

new vsphere.VirtualMachine(`test-vm`, {
    name: "test-vm",
    clone: {
        templateUuid: template.uuid,
    },
    extraConfig: {
        "guestinfo.userdata": userdata,
        "guestinfo.userdata.encoding": Encoding.Base64,
        "guestinfo.metadata": metadata,
        "guestinfo.metadata.encoding": Encoding.Base64,
    },
    networkInterfaces: [{
        networkId: "NETWORK_ID",
        adapterType: template.networkInterfaces[0].adapterType,
    }],
    disks: [{
        label: template.disks[0].label,
        size: template.disks[0].size,
        unitNumber: 0,
        datastoreId: "DATASTORE_ID",
    },{
        label: template.disks[1].label,
        size: template.disks[1].size,
        unitNumber: 1,
        datastoreId: "DATASTORE_ID",
    }],
    numCpus: 4,
    memory: 4096,
    cpuReservation: 1000,
    memoryReservation: 1024,
    datastoreId: "DATASTORE_ID",
    firmware: template.firmware,
    resourcePoolId: "RESOURCE_POOL_ID",
    folder: "/"
    guestId: template.guestId,
    waitForGuestIpTimeout: 10,
}, {
    ignoreChanges: [ "syncTimeWithHost", ],
    parent: parent,
});

@nithom
Copy link

nithom commented May 15, 2024

Regarding the exceptions for eptRviMode and hvMode, I found that the outputs in the stack state are just empty strings, no matter what you specify as input. When doing a pulumi refresh, the inputs are then set to empty strings as well. With that, the provider throws exceptions on later calls. Manually changing state (with pulumi stack export/import) such that inputs and outputs both contain the default values makes everything work normally again. But of course, that's not an ideal workaround.

@guineveresaenger
Copy link
Contributor

guineveresaenger commented May 17, 2024

hi @nithom and @cultpony -

per @mikhailshilkov 's suggestion, I've rolled an alpha release of the vsphere provider that includes the ability to add an environment variable that will allow the provider to use a new diffing strategy.

Could you run your local program using v4.11.0-alpha.1 and with

$ export PULUMI_ENABLE_PLAN_RESOURCE_CHANGE=true

set in the environment, and let us know if this issue persists? Thank you so much! 🙏

@guineveresaenger guineveresaenger added the awaiting-feedback Blocked on input from the author label May 20, 2024
@nithom
Copy link

nithom commented May 21, 2024

Hi @guineveresaenger , thanks for providing an alpha release. Unfortunately, the behavior stays the same with the new version and the environment variable set. After doing a pulumi refresh, the inputs and outputs of the two parameters still become empty strings in the stack state, and later pulumi up calls output a diff for the parameters.

@iwahbe iwahbe added needs-triage Needs attention from the triage team and removed awaiting-feedback Blocked on input from the author labels May 21, 2024
@guineveresaenger
Copy link
Contributor

Thank you for trying out the alpha version, @nithom. We'll take another look as soon as we can.

@iwahbe iwahbe removed the needs-triage Needs attention from the triage team label May 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug/diff kind/bug related to Pulumi generating wrong diffs on preview or up. kind/bug Some behavior is incorrect or out of spec
Projects
None yet
Development

No branches or pull requests

6 participants