Returning dynamic IP-address from VRA to Terraform

[leander091]

vRA 7.x version
7.4
Terraform version
v0.11.11
terraform-provider-vra7 plugin version
master

Describe the bug
I'm trying to execute the remote-exec provisioner on a machine created with the provider. The machine has created successfully but the provisioner fails because of the missing ip-address.

Is it possible pause the executing until VRA has received the ip-address?
I can update my environment after deployment with terraform refresh but that is not useful in this case.

To Reproduce
Steps to reproduce the behavior:

1. Terraform config file

resource_configuration = {        
        BP-Name.cpu  = "1"
        BP-Name.memory  = "2048"
        BP-Name.description = "testmachine"
        BP-Name.VirtualMachine.Network0.Name = "Network"
        BP-Name.ip_address = ""
    
     }
  connection {
    host     = "${self.resource_configuration.BP-Name.ip_address}"
    user     = "${var.ssh_user}"
    password = "${var.ssh_password}"
}

2. Terraform command
3. Error
   

Logs
Attach logs to help debug the issue

1. Attach vra-terraform.log
2. Attach Terraform console log (Enable terraform logs following the steps mentioned in https://www.terraform.io/docs/configuration/environment-variables.html)
3. crash.log(if any)

Desktop (please complete the following information):

• OS: Windows

[Prativa20]

@leander091 Yes, you are right, the ip_address is not assigned as soon as the resource creation is successful. Looking into this issue.

[Prativa20]

Hi @leander091 I was looking if terraform provides a way to add a pause or delay. I have found this
hashicorp/terraform#2689 and hashicorp/terraform#17726

It says you can write a local_exec provisioner.

Let me know if this helps your use case.

Thanks,
Prativa

[leander091]

Hello @Prativa20 Thanks for the update. I am going to add a local_exec provisioner to pause the executing until the VRA data collection has finished en the IP is returned.

I will keep you posted

EDIT I think there could be a cleaner option at the provider level. Maybe a optional option to mark the resource as provisioned if the IP-address is available in VRA and not before.

[mcascone]

I have this exact problem. Replace remote-exec with chef in @leander091's example and it would be my example. I am trying everything i can find to inject a delay after the machine comes up but does not have its ip yet, no luck yet.

[mcascone]

I have this exact problem. Replace remote-exec with chef in @leander091's example and it would be my example. I am trying everything i can find to inject a delay after the machine comes up but does not have its ip yet, no luck yet.

I figured it out. I am now using the FQDN of the new VM. The key was understanding that my instance of vRA is set up so that the VMs are called vSphere_Machine_1. It's right there in the UI, as Component Name.

There were definitely some mental leaps involved, and I'm sure I had to have seen all the other examples and made all the mistakes i did in order for my brain to put it all together at that moment. Whatever the mechanism, this code now WORKS:

in resource block

resource_configuration {
    vSphere_Machine_1.name = ""
  }

in provisioner block

    connection {
      host = "${self.resource_configuration.vSphere_Machine_1.name}"
     ....
}

Now, of course, on to the next brick wall... but I figured this out. And I intend to pay it forward to the community.

For others coming to this with the same question, you are looking for the Component Name in your vRa item's machine listing.

Another note: If you are creating multiple machines using count > 1, to output or otherwise access those machine names outside of that block, you have to splat the index like so:

output "machine-names" {
  # note the splatting with the "*", that's how we access the list of machines
  value = ["${vra7_deployment.TF-Deploy.*.resource_configuration.vSphere_Machine_1.name}"]
}

[hobovirtual]

Hi @Prativa20

Looks like the ip_address field in the resourceView is only loaded after a state collection. At least on vRealize Automation version 7.6

Could we use the NETWORK_ADDRESS from the NETWORK_LIST data map instead, which is available right away?

                "NETWORK_LIST": [
                    {
                        "componentTypeId": "com.vmware.csp.component.iaas.proxy.provider",
                        "componentId": null,
                        "classId": "dynamicops.api.model.NetworkViewModel",
                        "typeFilter": null,
                        "data": {
                            "NETWORK_ADDRESS": "10.10.10.100",
                            "NETWORK_MAC_ADDRESS": "00:50:56:a6:fa:c6",
                            "NETWORK_NAME": "vxw-dvs-24-universalwire-1-sid-10000-mgmt-10.10.10.0-24",
                            "NETWORK_NETWORK_NAME": "mgmt",
                            "NETWORK_PROFILE": "mgmt"
                        }
                    }
                ]

[hobovirtual]

FYI

Opened an issue on the terraform-providers git page

Thanks!

[akshatavijapur]

only terraform version with 0.11.8 and 0.11.13 are having the vra7 plugin as default. with the latest version or any other version of terraform will not work.

[mcascone]

only terraform version with 0.11.8 and 0.11.13 are having the vra7 plugin as default. with the latest version or any other version of terraform will not work.

I'm not sure what you're referring to here, could you please elaborate?

[danf425]

Hello,
Was there any fix to this?
I am able to get my machine's name, but not the IP address.
It seems the ipaddress doesn't appear for ~10 minutes after the machine is provisioned.

Is there a work around to connecting in order to do a remote-exec in the newly vRA provisioned machine?

[mcascone]

@danf425 that sounds like a DNS issue outside of terraform. Are you using the patterns shown above? Can you use <your-vra-blueprint-name>.name?

Also note that the resource_configuration syntax has changed in v0.12, it took me quite a while to sort it all out.
I'll update this with an example when I get a chance.

[danf425]

@mcascone Thanks for the quick reply!
Let me give you a bit more context. I am able to spin up the instance just fine.
My resource configuration is below:

resource_configuration = {
    CentosOS.description = "${var.deployment_description}"
    CentosOS.cpu = "${var.cpu}"
    CentosOS.memory = "${var.memory}"
    CentosOS.storage = "${var.storage}"
    CentosOS.ip_address = ""    
    CentosOS.primary_ip_address = ""
    CentosOS.name = ""
  }

This is my connection:

connection {
   type        = "ssh"
   host        = "${self.resource_configuration.CentosOS.name}"
   # host        = "${self.resource_configuration.CentosOS.ip_address}"
   user        = "vagrant"
   private_key = "${file("~/.ssh/vagrantdefaultkey")}"
   }
 }

I am able to get my machine's name. I've seen that some people were able to ssh with just this.
The problem is that my machine's name is not an fqdn (it looks something like: Machine001)

So my only option seems to be to get that ip_address. Even within vRA (The UI), the ip_address takes ~10 minutes to populate.

I'm currently trying to output the information without any connection in hopes that I can grab the ip with a terraform refresh after it's populated:

output "machine-name" {
  value = ["${vra7_deployment.automate_machine.*.resource_configuration.CentosOS.name}"]
}
output "ip-address" {
  value = ["${vra7_deployment.automate_machine.*.resource_configuration.CentosOS.ip_address}"]
}
output "primary-ip-address" {
  value = ["${vra7_deployment.automate_machine.*.resource_configuration.CentosOS.primary_ip_address}"]
}

So far no luck, but I'm hoping somebody has figured out a way to do all of this.
I know the syntax for the IP address is ip_address , but within vRA I see it as Primary IP Address .
Honestly at this point I'm just throwing darts and hoping something sticks.

[mcascone]

Can you connect to Machine001.yourdomain.com? Or is the name not linked to the DNS at all?

If it is, you could add the domain to the name string:

locals {
  full_name = join(".", list("${self.resource_configuration.CentosOS.name}", "yourdomain.com"))
}

connection {
   host        = "${local.full_name}"
}

I haven't tested this exact code, obviously, but I use the join in my TF code elsewhere.

As for the syntax change in TF 0.12,

resource_configuration = {
    CentosOS.description = "${var.deployment_description}"
    CentosOS.cpu = "${var.cpu}"
    CentosOS.memory = "${var.memory}"
    CentosOS.storage = "${var.storage}"
    CentosOS.ip_address = ""    
    CentosOS.primary_ip_address = ""
    CentosOS.name = ""
  }

Should become

resource_configuration = {
    "CentosOS.description" = var.deployment_description
    "CentosOS.cpu" = var.cpu
    "CentosOS.memory" = var.memory
    "CentosOS.primary_ip_address" = ""
     etc etc
  }

I've found that 0.12 doesn't enforce removing the interpolations, but it's preferred.

To access the resource_config items, this is from my code:

  resource_configuration = {
    "vSphere_Machine_1.name" = ""
    "vSphere_Machine_1.ip_address" = ""
    "vSphere_Machine_1.description" = "Terraform Machine ${count.index + 1}"
  }


connection {
      host = self.resource_configuration["vSphere_Machine_1.ip_address"]
}

output "Machine_Names" {
  # note the splat * if you are using `count`
  value = vra7_deployment.TF-Deploy[*].resource_configuration["vSphere_Machine_1.name"]
}

# Or, print the whole thing:
output "Machine_Info" {
  value = vra7_deployment.TF-Deploy.*.resource_configuration
}

This new syntax took me a while to nail down. Happy to share!

[danf425]

@mcascone
That's a great idea, and I gave it a try, but no dice.

My problem here seems to be with vRA/vSphere rather than my TF code, which is unfortunate.
For context, when a machine if provisioned in vRA, vRA doesn't know the IP right away.
I can go into vSphere, see the newly spun up machine and grab the IP address from there manually.
If I wait several minutes, vRA will finally get the IP address (you can finally see it on the UI)

My initial thought was that if I did a simple terraform refresh after I can see this information in the GUI, then I'd be able to fetch the ip address, but that's still not working. xxx.xxx.xxx.machine_name.ip_addresswill still return nothing

I'm currently exploring the possibility of spinning up an instance with a static IP as a workaround.

[mcascone]

It sounds like the IP is assigned, but doesn't propagate to vRA right away.
Can you get the ip of the new machine from the DNS from the command line, bypassing vRA? You might be able to use the local_exec provider to run an nslookup or similar if so.

[hobovirtual]

Hi @danf425

From what i understand, your virtual machine network is assigned by DHCP in your blueprint. If this is the case, then this behavior is normal. Since vRA doesn't assign the IP it won't be available to retrieve the IP until a State Collection is performed (every 15 minutes by default). This is explaining also the behavior your describing in the UI.

If you use a static IP, then you will be able to retrieve the IP right within your deployment, you can refer to @mcascone examples.

I would also suggest to use the latest and certified version of the provider which you can find here

There's an issue with the way vRA poulates the ip_adress metadata on version 7.6.0 which you can read here

In the case you wish to go with DNS, i've seen some issue with populating DNS (due to replication), you can add a sleep in your main.tf file if you want

provisioner "local-exec" {
  command = "sleep 120"
}