UDMP - Running the default wlan example fails

[ukd1]

I'm getting failures with the following code, mostly taken from the samples:

terraform {
  required_providers {
    unifi = {
      source = "paultyng/unifi"
      version = "0.13.1-beta.1"
    }
  }
}

provider "unifi" {
  username = "xxxxxx"
  password = "xxxxxx"
  api_url  = "https://192.168.86.1"
  allow_insecure = true
}

data "unifi_wlan_group" "default" {
}

data "unifi_user_group" "default" {
}

resource "unifi_wlan" "wifi" {
  name          = "myssid"
  vlan_id       = 10
  passphrase    = "12345678"
  wlan_group_id = data.unifi_wlan_group.default.id
  user_group_id = data.unifi_user_group.default.id
  security      = "wpapsk"
}

Here is the output of version + apply:

terraform % terraform version
Terraform v0.13.2
+ provider registry.terraform.io/paultyng/unifi v0.13.1-beta.1

terraform % terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

data.unifi_user_group.default: Refreshing state...
data.unifi_wlan_group.default: Refreshing state...

Error: unable to decode body: GET s/default/rest/wlangroup invalid character '<' looking for beginning of value

Error: unable to decode body: GET s/default/rest/usergroup invalid character '<' looking for beginning of value

FYI I'm running controller version 5.14.24 and Dream Machine Pro firmware 1.8.1-rc.3.

Thanks for writing this!

[paultyng]

I think the issue here is that your api_url should be in the form https://192.168.86.1/api/, but I'm not entirely sure (that is how mine looks locally at least, I'm using something like https://<ip address>:8443/api/).

[paultyng]

I should probably make sure the docs are clear about the path, I added some additional copy in 68386b3 that should be live with the next release.

[ukd1]

@paultyng awesome thanks, will retry shortly!

[paulhugill]

I think I have the same issue and believe it is related to the UDM Pro having different API paths.

There is a note about it here:
https://ubntwiki.com/products/software/unifi-controller/api

The login endpoint is /api/auth/login
All API endpoints need to be prefixed with /proxy/network

When I set the api_url to https://ip//proxy/network/api/, I no longer get the original error but I do get a 401 Unauthorized instead (probably because it has the wrong login endpoint now).

I'll see if I can get more detail but let me know what else I can provide, if needed.

Really looking forward to using this, thanks for putting it together!

[paultyng]

Hmm, yes please let me know, I only have the normal UDM, not the Pro, so unable to test this locally. I've tested with the controller in docker as well (the acceptance testing on this repository uses that).

[paulhugill]

Definitely something different compared to any examples I could find online (e.g. this page https://bartsimons.me/playing-around-with-the-ubiquiti-unifi-controller/), even just changing the api url's didnt help, so it might be a bigger issue.

This command let's me auth:
curl "https://ip/api/auth/login" -X POST -H "Content-Type:application/json" -d '{"username":"usernamegoeshere","password":"passwordgoeshere","strict":true}' --insecure -c cookies.txt

Then this lets me get the health and other endpoints follow this URL format:
curl --insecure -b cookies.txt -c cookies.txt 'https://ip/proxy/network/api/s/default/stat/health'

I'm afraid I'm not experienced with TF providers and can't really follow how it is calling anything, but happy to test anything out if you can point me in the right direction of where to adjust and then I could try to build it myself and test out.

Thanks again.

[paultyng]

What version of the controller do you have running? I can at least try that exact version in tests via docker possibly.

I'm asking someone at work if they can test with a UDM Pro as well, maybe get some additional info on this.

[paulhugill]

I realised earlier that there is an open issue on your SDK repo about this:
paultyng/go-unifi#1

I don't know Go though, so I don't think I'll be able to do a PR for any changes, but your idea of splitting out the login and api path variables probably makes the most sense.
Or maybe a bool variable which switches between the original and new paths.

I might have managed to make some hacky progress hard coding things in a new custom provider (never worked with custom providers or Go before, so all purely guesswork).
The Auth seemed to be failing until I added this for the header (in go-unifi/unifi/unifi.go), not sure if it is needed:
req.Header.Add("Content-Type", "application/json; charset=utf-8")

Now I have the plans working again (including data sources actually reading fully, which were failing originally) but it is failing to apply changes and I'm just getting Error: not found for everything.
Any other changes I've done, I think were just to build from my forks and hard code the change for all of the relative URLs, which we don't want to include anyway.

I have controller version 5.14.22.

[paultyng]

@paulhugill I think I have a PR up to fix this: #60

Assuming it passes the acc testing, I'll publishing a pre-release and you can give it a try.

[paulhugill]

Thanks @paultyng it is using the correct endpoints now but not logging in, just gives a 401 error (debug attached).

Putting req.Header.Add("Content-Type", "application/json; charset=utf-8") on line 154 of go-unifi/unifi/unifi.go seems to get it to authenticate and complete the plan, but on applying I get a Error: Not Found for every resource and a 404 response but it's showing the correct endpoint. I built a custom provider again with that change but that might be causing the 404 problem.

00 [DEBUG] plugin.terraform-provider-unifi.exe: -----------------------------------------------------
2020-09-25T03:20:57.019+0100 [DEBUG] plugin.terraform-provider-unifi.exe: 2020/09/25 03:20:56 [DEBUG] Unifi API Request Details:
2020-09-25T03:20:57.019+0100 [DEBUG] plugin.terraform-provider-unifi.exe: ---[ REQUEST ]---------------------------------------
2020-09-25T03:20:57.019+0100 [DEBUG] plugin.terraform-provider-unifi.exe: POST /proxy/network/api/s/default/rest/networkconf HTTP/1.1

2020-09-25T03:20:57.177+0100 [DEBUG] plugin.terraform-provider-unifi.exe: -----------------------------------------------------
2020-09-25T03:20:57.177+0100 [DEBUG] plugin.terraform-provider-unifi.exe: 2020/09/25 03:20:56 [DEBUG] Unifi API Response Details:
2020/09/25 03:20:57 [ERROR] eval: *terraform.EvalApplyPost, err: not found
2020-09-25T03:20:57.177+0100 [DEBUG] plugin.terraform-provider-unifi.exe: ---[ RESPONSE ]--------------------------------------
2020-09-25T03:20:57.177+0100 [DEBUG] plugin.terraform-provider-unifi.exe: HTTP/1.1 404 Not Found
2020-09-25T03:20:57.177+0100 [DEBUG] plugin.terraform-provider-unifi.exe: Content-Length: 9
2020-09-25T03:20:57.177+0100 [DEBUG] plugin.terraform-provider-unifi.exe: Accept-Ranges: bytes
2020-09-25T03:20:57.177+0100 [DEBUG] plugin.terraform-provider-unifi.exe: Connection: keep-alive
2020-09-25T03:20:57.177+0100 [DEBUG] plugin.terraform-provider-unifi.exe: Content-Type: text/plain; charset=utf-8
2020-09-25T03:20:57.177+0100 [DEBUG] plugin.terraform-provider-unifi.exe: Date: Fri, 25 Sep 2020 02:20:56 GMT
2020/09/25 03:20:57 [ERROR] eval: *terraform.EvalSequence, err: not found
2020-09-25T03:20:57.177+0100 [DEBUG] plugin.terraform-provider-unifi.exe: Strict-Transport-Security: max-age=15552000; includeSubDomains
2020-09-25T03:20:57.177+0100 [DEBUG] plugin.terraform-provider-unifi.exe: Vary: Origin
2020-09-25T03:20:57.177+0100 [DEBUG] plugin.terraform-provider-unifi.exe: X-Content-Type-Options: nosniff
2020-09-25T03:20:57.177+0100 [DEBUG] plugin.terraform-provider-unifi.exe: X-Csrf-Token: b55097d6-e2b9-4713-af1a-132d4c79106d
2020-09-25T03:20:57.177+0100 [DEBUG] plugin.terraform-provider-unifi.exe: X-Dns-Prefetch-Control: off
2020-09-25T03:20:57.177+0100 [DEBUG] plugin.terraform-provider-unifi.exe: X-Download-Options: noopen
in.terraform-provider-unifi.exe: X-Frame-Options: SAMEORIGIN
2020-09-25T03:20:57.187+0100 [DEBUG] plugin.terraform-provider-unifi.exe: X-Response-Time: 8ms
2020-09-25T03:20:57.187+0100 [DEBUG] plugin.terraform-provider-unifi.exe: X-Xss-Protection: 1; mode=block
2020-09-25T03:20:57.187+0100 [DEBUG] plugin.terraform-provider-unifi.exe:
2020-09-25T03:20:57.187+0100 [DEBUG] plugin.terraform-provider-unifi.exe: Not Found

In fact, I've tried a few PUT and POST commands with Curl or Postman, and all I get is a 404 Not Found, even though doing a GET to those same endpoints returns results.

Could I have missed a config setting to enable Writes with the API? I can't find anything and have been using the Owner or another SuperAdmin user for this.

I appreciate your help with this and I know you don't have a UDMP to test with but any suggestions would be welcome.

Thanks

[paultyng]

Maybe I need to ask Unifi to send me one on Twitter 😂

Someone on my team has a UDMP, I'll see if I can get her to try it out at some point.

The other thing you can do, logging in to the UI in the browser (via IP, not WebRTC), and just look at the URLs in the debug tools for some of the pages (like networks, etc). Save a network, create one, delete, etc. and grab those URLs.

[paulhugill]

I think you deserve one for all your effort putting this together (although I do understand there are some other problems with them!).

I might have found the cause of the 404's, good suggestion to look at a working one done manually.
Adding a x-csrf-token header seems to have it working, paths and everything otherwise seem fine.

Here is an example CURL that worked to great a FW Group (sanitized tokens obviously):

curl "https://<ipaddress>/proxy/network/api/s/default/rest/firewallgroup" --insecure \
  -H 'x-csrf-token: 3bsdkfjhskjhdkfjhskjh08a009' \
  -H 'Cookie: TOKEN=eyJhbGciOdksfjkjkhsjkhfd4kZyb8268GSAWbhRA' \
  -d '{"group_members":["ip1","ip2"],"group_type":"address-group","name":"test"}'

I don't know how that gets fed in but it does look like that header is also on the GET requests as well, so hopefully not a big deal to add. Guess they just don't care about that for Read-Only calls, which makes sense.

[chrishas35]

I'm seeing the same 401 response with 0.15.0-beta.1

[chrishas35]

I was able to confirm that adding the Content-Type: application/json header resolves the 401 and once past that I am seeing the same issues related to CSRF. Since the initial addition was based on unifi-poller, I went and looked at how they are handling and found the following commit: unpoller/unifi@0e3f394. Looks like both these changes will need to land in go-unifi.

[paulhugill]

I was able to add the CSRF pieces from that commit, into my fork and then get the apply to work properly with a custom built version of the provider, thanks @chrishas35.

@paultyng would you like me to do a PR with the changes? I don't have any way to test it doesn't break the other controller versions, just that it seems to work on the UDMP.

[chrishas35]

I've opened a PR against paultyng/go-unifi to add the two required headers for UDM Pro.

[paultyng]

Thanks! Will continue the discussion over in go-unifi.

[paultyng]

I added the CSRF support and just cut 0.15.0-beta.2 (should be live shortly): https://github.com/paultyng/terraform-provider-unifi/releases/tag/v0.15.0-beta.2

terraform {
  required_providers {
    unifi = {
      source = "paultyng/unifi"
      version = "0.15.0-beta.2"
    }
  }
}

Let me know if this fixes it!

[paulhugill]

@paultyng This seems to be working well for me now.
Thanks for creating it, so much better to manage via code than poking around via the GUI.

@chrishas35 Thanks for sorting the CSRF piece.

[paultyng]

Great will close this and I can ship a real 0.15.0 soon.