New-NsxLoadBalancerPool does not work (including a fix)

[StefanPahrmann]

When calling New-NsxLoadBalancerPool, following is returned: "invoke-nsxwebrequest : Invoke-NsxWebRequest : The NSX API response received indicates a failure. 400 : Bad Request". Also 3 times this error occours: "The property 'OwnerDocument' cannot be found on this object"

It happened for both using IP-address or VM-name as poolmembers.

Following changes was made in New-NsxLoadBalancerPool ('#' indicates the old settings) :

#[System.XML.XMLElement]$xmlPool = $_LoadBalancer.OwnerDocument.CreateElement("pool")
  [System.XML.XmlDocument]$xmldoc = New-Object System.XML.XmlDocument
  [System.XML.XMLElement]$xmlPool = $xmldoc.CreateElement("pool")  
#$_LoadBalancer.appendChild($xmlPool) | out-null
  $xmldoc.appendChild($xmlPool) | out-null

#$URI = "/api/4.0/edges/$EdgeId/loadbalancer/config"
$URI = "/api/4.0/edges/$EdgeId/loadbalancer/config/pools"

#$null = invoke-nsxwebrequest -method "put" -uri $URI -body $body -connection $connection
$null = invoke-nsxwebrequest -method "post" -uri $URI -body $body -connection $connection

Source:
Official NSX 6.3 API p. 325+326

Regards
Stefan

[alagoutte]

Hi Stefan,

i think, it will be better to push directly a PR with your fix

(and we really need to add some test on Load Balancer...)

[StefanPahrmann]

Hi again,
just tried that now (was my first time ...) - could be great if you could check if it was done properly. I will definitely look more into the loadbalancer code, since I need to migrate a lot from a running HAProxy.
Thanks

[nmbradford]

hi @neoxinth - thanks for this! Before I get into reviewing the PR, can you include the details of the error you are getting including the exact syntax of getting/creating the monitor, defining the members etc?

While you are correct - the documented (and more efficient) method of crud'ing pools is via /api/4.0/edges/$EdgeId/loadbalancer/config/pools, its is also perfectly valid to modify the base edge xml as the existing LB cmdlets do. (The reason for this is historical as the LB bits of PowerNSX are pretty long in the tooth now and in sore need of attention!)

I need to validate exactly what you are seeing as I've done a quick test in the lab against 633 with the current cmdlets but I can't reproduce your issue.

My main reluctance around just merging #435 is that (as @alagoutte points out) we dont have test coverage for the LB cmdlets and without this, its very easy to break PowerNSX against older versions of NSX without realising it. To resolve this - we have two options. 1) wait until someone (probably me or @pandom if no one else steps up ;) ) writes some tests for LB, 2) Hope that we dont break things. At the moment I like option 1.

[nmbradford]

I should have added - if you are up to it - feel free to have a stab at writing some LB tests - at a minimum, some basic ones around pool creation would be enough to test a merge of #435. Take a look at https://powernsx.github.io/get-started#contributing for more info.

[StefanPahrmann]

Hi,
Just for sake of completeness, I'm using the stable-build (1054), so maybe you've already fixed it :). I've never really worked with github before (except for downloading projects). I'm also no developer, so I'm not sure how formal testing is done (I will try to look into it though).

The commands I ran are following:

$vm1="myvm"
$name="my_backend"
$poolmember=New-NsxLoadBalancerMemberSpec -name $vm1 -Member (get-vm $vm1) -MonitorPort 9200 -port 9200
$monitor = Get-NsxEdge LB-Util-01 |  Get-NsxLoadBalancer| Get-NsxLoadBalancerMonitor default_tcp_monitor
$LbAlgo = "round-robin"

Until here everything goes well, objects are populated, I can see that monitors and poolmembers are defined.

Get-NsxEdge LB-Util-01 | Get-NsxLoadBalancer | New-NsxLoadBalancerPool -name $name -Transparent:$false -Algorithm $LbAlgo -Memberspec $poolmember -Monitor $monitor
The error is as follows:

invoke-nsxwebrequest : Invoke-NsxWebRequest : The NSX API response received indicates a failure. 500 : Internal Server
Error : Response Body: <?xml version="1.0" encoding="UTF-8"?>
<error><errorCode>100</errorCode></error>
At C:\Program Files\WindowsPowerShell\Modules\PowerNSX\3.0.1054\PowerNSX.psm1:30284 char:17
+ ...     $null = invoke-nsxwebrequest -method "put" -uri $URI -body $body  ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidResult: (Invoke-NsxWebRequest:String) [Invoke-NsxWebRequest], InternalNsxApiExcep
   tion
    + FullyQualifiedErrorId : NsxAPIFailureResult,Invoke-NsxWebRequest

The property 'OwnerDocument' cannot be found on this object. Verify that the property exists.
At C:\Program Files\WindowsPowerShell\Modules\PowerNSX\3.0.1054\PowerNSX.psm1:532 char:5
+     [System.XML.XMLElement]$xmlNode = $xmlRoot.OwnerDocument.CreateEl ...
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], PropertyNotFoundException
    + FullyQualifiedErrorId : PropertyNotFoundStrict

The property 'OwnerDocument' cannot be found on this object. Verify that the property exists.
At C:\Program Files\WindowsPowerShell\Modules\PowerNSX\3.0.1054\PowerNSX.psm1:533 char:5
+     [System.XML.XMLNode]$xmlText = $xmlRoot.OwnerDocument.CreateTextN ...
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], PropertyNotFoundException
    + FullyQualifiedErrorId : PropertyNotFoundStrict

You cannot call a method on a null-valued expression.
At C:\Program Files\WindowsPowerShell\Modules\PowerNSX\3.0.1054\PowerNSX.psm1:534 char:5
+     $xmlNode.AppendChild($xmlText) | out-null
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull

You cannot call a method on a null-valued expression.
At C:\Program Files\WindowsPowerShell\Modules\PowerNSX\3.0.1054\PowerNSX.psm1:535 char:5
+     $xmlRoot.AppendChild($xmlNode) | out-null
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull

[alagoutte]

What NSX release do you are using ?

[StefanPahrmann]

6.3.4

[nmbradford]

ah - ok, its a 500, not 400 :) different response to your initial query. Ok - I have repro'ed now, but the build from scratch script that uses those same cmdlets still works, so I'm missing something. Ill try to take a look next week.

[StefanPahrmann]

Sorry, I think the output is from an already altered module. You can see the error below (which looks pretty much the same, except for the error 500)
I think I will try to rebuild an environment from scratch with the a clean master-branch-clone, just to be sure you see the right picture.
Information that might is relevant is, that I'm using the vcenter-based SSO-logon and that it has been upgraded from 6.3.1 to 6.3.2 to 6.3.4

The error I saw originally is this:

PS C:\> Get-NsxEdge LB-Util-01 |  Get-NsxLoadBalancer | New-NsxLoadBalancerPool -name $name -Transparent:$
false -Algorithm $LbAlgo -Memberspec $poolmember -Monitor $monitor
invoke-nsxwebrequest : Invoke-NsxWebRequest : The NSX API response received indicates a failure. 500 : Internal Server
Error : Response Body: <?xml version="1.0" encoding="UTF-8"?>
<error><errorCode>100</errorCode></error>
At C:\Program Files\WindowsPowerShell\Modules\PowerNSX\3.0.1054\PowerNSX.psm1:30284 char:17
+ ...     $null = invoke-nsxwebrequest -method "put" -uri $URI -body $body  ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidResult: (Invoke-NsxWebRequest:String) [Invoke-NsxWebRequest], InternalNsxApiExcep
   tion
    + FullyQualifiedErrorId : NsxAPIFailureResult,Invoke-NsxWebRequest

The property 'OwnerDocument' cannot be found on this object. Verify that the property exists.
At C:\Program Files\WindowsPowerShell\Modules\PowerNSX\3.0.1054\PowerNSX.psm1:532 char:5
+     [System.XML.XMLElement]$xmlNode = $xmlRoot.OwnerDocument.CreateEl ...
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], PropertyNotFoundException
    + FullyQualifiedErrorId : PropertyNotFoundStrict

The property 'OwnerDocument' cannot be found on this object. Verify that the property exists.
At C:\Program Files\WindowsPowerShell\Modules\PowerNSX\3.0.1054\PowerNSX.psm1:533 char:5
+     [System.XML.XMLNode]$xmlText = $xmlRoot.OwnerDocument.CreateTextN ...
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], PropertyNotFoundException
    + FullyQualifiedErrorId : PropertyNotFoundStrict

You cannot call a method on a null-valued expression.
At C:\Program Files\WindowsPowerShell\Modules\PowerNSX\3.0.1054\PowerNSX.psm1:534 char:5
+     $xmlNode.AppendChild($xmlText) | out-null
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull

You cannot call a method on a null-valued expression.
At C:\Program Files\WindowsPowerShell\Modules\PowerNSX\3.0.1054\PowerNSX.psm1:535 char:5
+     $xmlRoot.AppendChild($xmlNode) | out-null
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull

[StefanPahrmann]

Hi again,
finally I build my (clean) test-environment and tried to run the commands, but I figured that I've forgot something in the PR.

As it turns out, it's in the current build not possible to create a pool with objects (as it will complain that no IP is available:

PS C:\> Get-NsxEdge LB-01 | Get-NsxLoadBalancer | New-NsxLoadBalancerPool -name $name -Transparent:$fa
lse -Algorithm $LbAlgo -Memberspec $poolmember -Monitor $monitor
New-NsxLoadBalancerPool : Cannot validate argument on parameter 'MemberSpec'. XML Element specified does not contain an
 ipAddress property.  Create with New-NsxLoadbalancerMemberSpec
At line:1 char:134
+ ... Transparent:$false -Algorithm $LbAlgo -Memberspec $poolmember -Monito ...
+                                                       ~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [New-NsxLoadBalancerPool], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationError,New-NsxLoadBalancerPool

So I've also created a change on line 2796 which allows objects as poolmembers from:

        if ( -not ( $argument | get-member -name ipAddress -Membertype Properties)) {
                throw "XML Element specified does not contain an ipAddress property.  Create with New-NsxLoadbalancerMemberSpec"
        }

to

        if ( -not ( $argument | get-member -name ipAddress -Membertype Properties)) {
            if ( -not ( $argument | get-member -name groupingObjectId -MemberType Properties ) ) {
                throw "XML Element specified does not contain an ipAddress property.  Create with New-NsxLoadbalancerMemberSpec"
            }
        }

(Note the use of the groupingObjectID, which defines a vcenter-object)

I haven't done a PR on this yet, since I'm not sure whether I'm testing on the right version, since you don't get the same error and can't reproduce it?

Regards
Stefan

[StefanPahrmann]

I did a PR on this one and tested it's functionality and it looks good.

Running the "start-test edge" applet didn't show any error.

[alagoutte]

Hi neoxinth,

The problem, there is no yet LB test on Edge testsuite...

[StefanPahrmann]

Hmm, I can see that. Is there a way to build a test-suite more or less automatically? I could help with looking into it then, but I'm absolutely not experienced with it.

[alagoutte]

You can look on tests/integration folder
it using pstester (it is easy to use)
you can look some of my PR (like #426 )

[alagoutte]

Hi @neoxinth
I found time to write some LB test... #597 and #599

May be need to look for add your test...

[Nick-Andreev]

While you are correct - the documented (and more efficient) method of crud'ing pools is via /api/4.0/edges/$EdgeId/loadbalancer/config/pools, its is also perfectly valid to modify the base edge xml as the existing LB cmdlets do. (The reason for this is historical as the LB bits of PowerNSX are pretty long in the tooth now and in sore need of attention!)

@nmbradford is probably no longer monitoring this, but just to close this off, it's happening under a very specific conditions. That's why you couldn't recreate it.

You have to already have at least one LB VIP configured to cause this issue. Why - I don't know. Maybe some bug in NSX REST API?

This is how you can reproduce it. This code will succeed:

$WebMember1 = New-NsxLoadBalancerMemberSpec -name Web01 -IpAddress 192.168.200.11 -Port 80
$lb_pool = Get-NsxEdge -Name $lbedge1name | Get-NsxLoadBalancer | New-NsxLoadBalancerPool -Name Test -Algorithm Round-Robin -MemberSpec $WebMember1
$lb_app_profile = Get-NsxEdge $lbedge1name | Get-NsxLoadBalancer | New-NsxLoadBalancerApplicationProfile -Name pester_lb_app_profile -Type http
Get-NsxEdge -Name $lbedge1name | Get-NsxLoadBalancer | New-NsxLoadBalancerPool -Name Test2 -Algorithm Round-Robin -MemberSpec $WebMember1

But this will fail on creating the second pool:

$WebMember1 = New-NsxLoadBalancerMemberSpec -name Web01 -IpAddress 192.168.200.11 -Port 80
$lb_pool = Get-NsxEdge -Name $lbedge1name | Get-NsxLoadBalancer | New-NsxLoadBalancerPool -Name Test -Algorithm Round-Robin -MemberSpec $WebMember1
$lb_app_profile = Get-NsxEdge $lbedge1name | Get-NsxLoadBalancer | New-NsxLoadBalancerApplicationProfile -Name pester_lb_app_profile -Type http
Get-NsxEdge -Name $lbedge1name | Get-NsxLoadBalancer | Add-NsxLoadBalancerVip -name "pester_vip" -Description "Pester VIP" -ipaddress 1.1.1.1 -Protocol http -Port 80 -ApplicationProfile $lb_app_profile -DefaultPool $lb_pool -AccelerationEnable
Get-NsxEdge -Name $lbedge1name | Get-NsxLoadBalancer | New-NsxLoadBalancerPool -Name Test2 -Algorithm Round-Robin -MemberSpec $WebMember1

The way to fix it is to make sure you include the pool ID in the body of the REST call, like this:
<poolId>pool-2</poolId>

Current implementation of the New-NsxLoadBalancerPool doesn't do that. But rather than trying to figure out how to generate a pool ID (I don't know if you can do that programmatically, I just tested it manually in Postman) and use /api/4.0/edges/$EdgeId/loadbalancer/config, I think it's better to use the more specific REST API call /api/4.0/edges/$EdgeId/loadbalancer/config/pools, how @neoxinth did in this PR. It doesn't require a pool ID.

[alagoutte]

Thanks @Nick-Andreev for explain

Can you add new test and a push a fix ? or you need some help ?

[Nick-Andreev]

@alagoutte, this pull request goes back to 2017 and far behind the current build. Would you recommend creating a new PR? Or is there a better way to address this?

[alagoutte]

Kept this issue and create a new PR with the fix