HTTP 400 Bad Request\n - cvc-complex-type.2.4.a: Invalid content was found starting with element 'CustomizationScript' when updating guest customization

[anilkapu]

Hi,

I'm using pyvcloud to update computer_name via guest customization for few Windows machines. But I get the following error when I try to do the same for Windows server editions like Windows 2012 server/ Windows 2016 server/Windows 2016 server-specter/Windows 2019 server
{code}
fatal: [win-xagt5 -> localhost]: FAILED! => {"changed": false, "msg": "Status code: 400/BAD_REQUEST, [ 3196610e-a0c0-4235-a093-6c1347753bc5 ] HTTP 400 Bad Request\n - cvc-complex-type.2.4.a: Invalid content was found starting with element 'CustomizationScript'. One of '{"http://www.vmware.com/vcloud/v1.5\":Link, WC[##other:"http://www.vmware.com/vcloud/v1.5"]}' is expected. (request id: 3196610e-a0c0-4235-a093-6c1347753bc5)"}
{code}
I can successfully update for any other windows/linux/OSX VMs.

I also see same issue if I try to enable Admin Password by passing parameters admin_password_enabled=True and admin_password={str}
pyvcloud version: 21.0.0
vcloud Director api ver: 31.0

My code is following the test https://github.com/vmware/pyvcloud/blob/d9100f8a23b10659b382338fefe670f4e30131c4/system_tests/vm_tests.py#L683

Also I could successfully update computer_name for above mentioned OSs when using pyvcloud version 18.2.2

[anilkapu]

can any one help on this? Error is very cryptic and not giving proper information as to what is wrong in the request

[shashim22]

If I understand correctly, updates are successful in 18.2.2 for Windows 2012 server/ Windows 2016 server/Windows 2016 server-specter/Windows 2019 server.

Could you please enable pycloud logs and compare the request body between 18.2.2 and 21.0.0?

[artw]

I have a similar problem, but with EdgeGateway. I'm trying to set Rate Limits for a Gateway Interface. After some digging, I found the reason. Yet to find a workaround.

vCloud Director API request parser expects a specific attribute order. pyvcloud messes this order up somehow. In my specific case, the request fails because of the attribute marked by an arrow below. Why it got there, no idea.

        <GatewayInterfaces>
            <GatewayInterface>
                <Name>Public</Name>
                <DisplayName>Public</DisplayName>
                <Network href="https://10.204.3.23/api/admin/network/34ba0e75-dd63-44ad-ae06-c24a1cc5774d"
                         id="urn:vcloud:network:34ba0e75-dd63-44ad-ae06-c24a1cc5774d" name="Public"
                         type="application/vnd.vmware.admin.network+xml"/>
                <InterfaceType>uplink</InterfaceType>
                <SubnetParticipation>
                    <Gateway>192.168.1.1</Gateway>
                    <Netmask>255.255.255.0</Netmask>
                    <SubnetPrefixLength>24</SubnetPrefixLength>
                    <IpAddress>192.168.1.105</IpAddress>
                    <UseForDefaultRoute>true</UseForDefaultRoute>
                </SubnetParticipation>
                <ApplyRateLimit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                                xmlns:xs="http://www.w3.org/2001/XMLSchema">true
                </ApplyRateLimit>
  ---->     <UseForDefaultRoute>true</UseForDefaultRoute>
                <InRateLimit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                             xmlns:xs="http://www.w3.org/2001/XMLSchema">50.0
                </InRateLimit>
                <OutRateLimit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                              xmlns:xs="http://www.w3.org/2001/XMLSchema">50.0
                </OutRateLimit>
            </GatewayInterface>
        </GatewayInterfaces>

[artw]

A follow up to my previous comment.
By experimenting with the code and using pickle to dump the objects to files, I see that the attribute ordering is defined by some internal stuff in lxml.objectify object. It always puts UseForDefaultRoute after ApplyRateLimit for some unknown reason...

[artw]

found a hacky workaround to reorder the elements so the vCD accepts the request.

       is_def_route = gateway_inf.UseForDefaultRoute.text
        gateway_inf.remove(gateway_inf.UseForDefaultRoute)
        gateway_inf.UseForDefaultRoute = E.UseForDefaultRoute(is_def_route)

I'm sure this will bite again, the proper solution is either fix vCD deserializer to accept any order of elements. Or rewrite the pyvcloud part that uses lxml to always use the order that vCD expects.

[mario-vm]

I found the same problem while trying to update AdminPassword field. As @artw explained the problem is that vCD expects an specific order in the fields, however that order is messed up by pyvcloud. When updating guest customization first a get request is performed, however that get request does not have AdminPasswordField (is not set) then pyvcloud just adds the field at the end of the fields obtained via get which is not the right place.

1. The value retrieved via get, does not contain AdminPassword field

<GuestCustomizationSection xmlns:root="http://www.vmware.com/vcloud/v1.5" xmlns:ns0="http://schemas.dmtf.org/ovf/envelope/1">
   <ns0:Info>Specifies Guest OS Customization Settings</ns0:Info>
   <Enabled>true</Enabled>
   <ChangeSid>false</ChangeSid>
   <VirtualMachineId>0a6da371-51ac-4d14-a1df-a2ec8c67776f</VirtualMachineId>
   <JoinDomainEnabled>false</JoinDomainEnabled>
   <UseOrgSettings>false</UseOrgSettings>
   <AdminPasswordEnabled>true</AdminPasswordEnabled>
   <AdminPasswordAuto>false</AdminPasswordAuto>
   <AdminAutoLogonEnabled>false</AdminAutoLogonEnabled>
   <AdminAutoLogonCount>0</AdminAutoLogonCount>
   <ResetPasswordRequired>false</ResetPasswordRequired>
   <ComputerName>vm-centos8</ComputerName>
   <Link href="https://cloud-domain/api/vApp/vm-0a6da371-51ac-4d14-a1df-a2ec8c67776f/guestCustomizationSection/" type="application/vnd.vmware.vcloud.guestCustomizationSection+xml" rel="edit" />
</GuestCustomizationSection>

2. The value generated by pyvcloud, note AdminPassword field at the end

<GuestCustomizationSection xmlns:root="http://www.vmware.com/vcloud/v1.5" xmlns:ns0="http://schemas.dmtf.org/ovf/envelope/1">
   <ns0:Info>Specifies Guest OS Customization Settings</ns0:Info>
   <Enabled>true</Enabled>
   <ChangeSid>false</ChangeSid>
   <VirtualMachineId>0a6da371-51ac-4d14-a1df-a2ec8c67776f</VirtualMachineId>
   <JoinDomainEnabled>false</JoinDomainEnabled>
   <UseOrgSettings>false</UseOrgSettings>
   <AdminPasswordEnabled>true</AdminPasswordEnabled>
   <AdminPasswordAuto>false</AdminPasswordAuto>
   <AdminAutoLogonEnabled>false</AdminAutoLogonEnabled>
   <AdminAutoLogonCount>0</AdminAutoLogonCount>
   <ResetPasswordRequired>false</ResetPasswordRequired>
   <ComputerName>vm-centos8</ComputerName>
   <Link href="https://cloud-domain/api/vApp/vm-0a6da371-51ac-4d14-a1df-a2ec8c67776f/guestCustomizationSection/" type="application/vnd.vmware.vcloud.guestCustomizationSection+xml" rel="edit" />
    <AdminPassword>xxxxxxx</AdminPassword>
</GuestCustomizationSection>

3. The value that pyvcloud should generate:

<GuestCustomizationSection xmlns:root="http://www.vmware.com/vcloud/v1.5" xmlns:ns0="http://schemas.dmtf.org/ovf/envelope/1">
   <ns0:Info>Specifies Guest OS Customization Settings</ns0:Info>
   <Enabled>true</Enabled>
   <ChangeSid>false</ChangeSid>
   <VirtualMachineId>0a6da371-51ac-4d14-a1df-a2ec8c67776f</VirtualMachineId>
   <JoinDomainEnabled>false</JoinDomainEnabled>
   <UseOrgSettings>false</UseOrgSettings>
   <AdminPasswordEnabled>true</AdminPasswordEnabled>
   <AdminPasswordAuto>false</AdminPasswordAuto>
   <AdminPassword>xxxxxxx</AdminPassword>
   <AdminAutoLogonEnabled>false</AdminAutoLogonEnabled>
   <AdminAutoLogonCount>0</AdminAutoLogonCount>
   <ResetPasswordRequired>false</ResetPasswordRequired>
   <ComputerName>vm-centos8</ComputerName>
   <Link href="https://cloud-domain/api/vApp/vm-0a6da371-51ac-4d14-a1df-a2ec8c67776f/guestCustomizationSection/" type="application/vnd.vmware.vcloud.guestCustomizationSection+xml" rel="edit" />
</GuestCustomizationSection>

I am pretty sure that there are more problems like this in pvcloud, the problem is that pvcloud adds missing fields to the end and that might not be the expected place for vCD.