Support for vehicle command proxy / FleetAPI

[RichieB2B]

The legacy OwnerAPI will soon be dropped by Tesla. Newer cars will support only signed commands using the Vehicle Command protocol. Pre-2021 Model X/Y will move to the FleetAPI. This PR supports both. For the Vehicle Command protocol it uses the proxy supplied by Tesla.

This PR adds 3 new configuration settings:

1. teslaApiUrl with as default the FleetAPI URL for North America. If this setting is missing in the config, the old OwnerAPI URL is used. This ensures that TWCManager keep working as-is with existing configurations.
2. teslaApiClientID is the client_id of the app registered with Tesla. This is needed to refresh the tokens.
3. httpProxyCert should point to the certificate of the proxy if teslaApiUrl is set to a proxy URL.

Fixes #545

[MikeBishop]

Just to clarify, this still requires a public endpoint and developer account for each user, correct?

[MikeBishop]

Relieved to see that the diff is so small. Thanks for putting this together.

[RichieB2B]

Just to clarify, this still requires a public endpoint and developer account for each user, correct?

A Tesla developer account is needed to register the app with Tesla. During this process the public key for the app is loaded from a public website. So there will be a shakeout of home use of the API because this is certainly not for everyone.

[RichieB2B]

Yes, this surprised me as well. The Fleet API is basically the same as the Owner API except it needs a token approved by both Tesla (by registering an app) and the user (by using the Tesla app). The Vehicle Command protocol is hidden away behind a proxy which understands the Fleet API. A real elegant solution that makes the transition quite easy.

[ChutneyMary]

During this process the public key for the app is loaded from a public website.

Just to clarify, I would need to set up a public facing website hosted on my Raspberry Pi, open appropriate ports for said website, and hope my home network is not compromised by the onslaught of attacks? Are there any guides you can suggest for users to safely do the above? I'm not sure if I understand how to host a public key.

So there will be a shakeout of home use of the API because this is certainly not for everyone.

Agreed... Although any assistance/guidance would be appreciated to do this with minimum 'exposure'.

[ngardiner]

Many thanks for this @RichieB2B

Just one concern around the certificate verification logic. If using the local proxy, self.proxyCert is set to the cert specified. If using fleet api and commented out, it will be set to None. I can't find any definitive explanation of what verify=None will do, it is a little bit too close to verify=False (which disables SSL validation) for comfort.

I think this could be easily fixed with using this instead:

self.proxyCert = config["config"].get("httpProxyCert", True)

I believe Tesla are actively disabling segments of the owner-api as we speak, in particular the vehicles endpoint.

[RichieB2B]

You're right. The default should be True not None. I'll push a fix.

https://github.com/psf/requests/blob/96b22fa18c00831656ee4b286bf1c9062459b00a/src/requests/sessions.py#L416-L424

[RichieB2B]

verify=None indeed disabled the certificate verification. Horrible idea to mix a boolean and string but it is well documented. Good catch, I totally assumed setting a string variable to None would fail over to the default of using the supplied certificate bundle. This is not the case:

https://github.com/psf/requests/blob/96b22fa18c00831656ee4b286bf1c9062459b00a/src/requests/adapters.py#L249-L274

[RichieB2B]

None does seem to enable certificate validation after all, see psf/requests#6624

[RichieB2B]

It seems on the Owner API vehicles endpoint has today been disabled by Tesla. TeslaMate found an easy fix already. This will help a lot of TWCManager users that are not developers. I'll make PR for it.

[RichieB2B]

During this process the public key for the app is loaded from a public website.

Just to clarify, I would need to set up a public facing website hosted on my Raspberry Pi, open appropriate ports for said website, and hope my home network is not compromised by the onslaught of attacks? Are there any guides you can suggest for users to safely do the above? I'm not sure if I understand how to host a public key.

No, you just need to host the generated public key (a text file) on a public website. There is absolutely no need to run TWCManager on the public internet.

So there will be a shakeout of home use of the API because this is certainly not for everyone.

Agreed... Although any assistance/guidance would be appreciated to do this with minimum 'exposure'.

I have recorded all the steps during registration. I'll make a write up when I have more time.

[ngardiner]

One thing nagging at me is whether we can auto-detect the best URL to use. We'll soon be forced to move all API calls over to this API and having people hardcode their geo DNS entry just feels a bit messy (and who knows what will happen down the track when Tesla changes their minds)

If we were to do it, any thoughts how? I had one or two - calling a geolocation API based on source IP or maybe a DNS lookup that could be used, but an even easier option might be to make HTTPS calls to a list of the API endpoints and see which one responds fastest? Just some thoughts.

[RichieB2B]

To use the Fleet API (for older Model S/X without using a proxy) you still need to register an app with Tesla and fill in the teslaApiClientID in config.json. Since this is a manual step anyway, setting teslaApiUrl to the correct region is a small thing to ask.

[RichieB2B]

One way to automate setting the correct region URL is to decode the API token using PyJWT.

[RichieB2B]

Decoding the token is not very helpful:

>>> token=settings['carApiRefreshToken']
>>> decoded = jwt.decode(token, options={"verify_signature": False, "verify_aud": False})
>>> decoded['data']['aud']
['https://fleet-api.prd.na.vn.cloud.tesla.com', 'https://fleet-api.prd.eu.vn.cloud.tesla.com']
>>> token=settings['carApiBearerToken']
>>> decoded = jwt.decode(token, options={"verify_signature": False, "verify_aud": False})
>>> decoded['aud']
['https://fleet-api.prd.na.vn.cloud.tesla.com', 'https://fleet-api.prd.eu.vn.cloud.tesla.com', 'https://auth.tesla.com/oauth2/v3/userinfo']

[RichieB2B]

Or maybe the ou_code points to the region?

>>> token=settings['carApiBearerToken']
>>> decoded = jwt.decode(token, options={"verify_signature": False, "verify_aud": False})
>>> decoded['ou_code']
'EU'

[MikeBishop]

Just to clarify, I would need to set up a public facing website hosted on my Raspberry Pi, open appropriate ports for said website, and hope my home network is not compromised by the onslaught of attacks? Are there any guides you can suggest for users to safely do the above? I'm not sure if I understand how to host a public key.

Agreed... Although any assistance/guidance would be appreciated to do this with minimum 'exposure'.

The device publishing the key need not be the same device running TWCManager, they just need to have the same key. You could host the key on any Internet-facing device or service, so long as the domain name you use resolves to that endpoint. (A dynamic DNS name from afraid.org or similar might be sufficient, for example.) Static hosting of one file isn't exactly a big attack surface.

[RichieB2B]

According to their documentation Tesla does the verification on TLD only, not FQDN. I would not recommend using a domain name you do not own. OTOH only you have the private key so there is little chance of abuse.

[mshoe007]

Just to clarify, I would need to set up a public facing website hosted on my Raspberry Pi, open appropriate ports for said website, and hope my home network is not compromised by the onslaught of attacks? Are there any guides you can suggest for users to safely do the above? I'm not sure if I understand how to host a public key.

I set up a developer account (and hosted the public key on my public server) using this tutorial

https://shankar-k.medium.com/tesla-developer-api-guide-account-setup-app-creation-registration-and-third-party-da24aba1bddd

[MikeBishop]

I've created an application that can help with this, for anyone comfortable spinning up a Docker container. Anyone willing to take https://github.com/MikeBishop/fleetapi-tokens for a spin and give me suggestions, I'd appreciate it.

[ChutneyMary]

I would love to help but I still can't get an account set up on the Developer portal. Has anyone had issues? I usually get 'Can't onboard app'. Any suggestions?

[MikeBishop]

When you're trying to register an app, or earlier?

[ChutneyMary]

At the screen where the API & Scopes is checked (Profile Information, Vehicle Information, Vehicle Commands etc.). The Submit button results in an instantaneous message - 'Unable to onboard app'.

[MikeBishop]

I've found that it rejects certain app names - try something that doesn't include the word Tesla?

[ChutneyMary]

Yes, I've given that a go without success. What are you selecting from the options - 'Authorization Code and Machine-to-Machine' or 'Machine-to-Machine only'?

[MikeBishop]

That doesn't ring a bell. For scopes, I'm selecting everything. Maybe send a screenshot? If you'd prefer DM, I'm on Discord as @evequefou.

[ChutneyMary]

The options - 'Authorization Code and Machine-to-Machine' or 'Machine-to-Machine only' appear if USA is selected as the country. My country selection (Australia) doesn't offer the option.
I've discussed the problem with creating an account on other forums. It seems to be an issue for others as well.
The suspicion is that Telsa hasn't provisioned enough server capacity and/or mis-configured local servers.

[RichieB2B]

At the screen where the API & Scopes is checked (Profile Information, Vehicle Information, Vehicle Commands etc.). The Submit button results in an instantaneous message - 'Unable to onboard app'.

When you click submit it tries to download the public key from your website. But I think the error message was different when it is missing.

I have read somewhere the app names must be the unique.

[alexeiw123]

With this merged, are we able to update to latest version and behaviour is returned or are we required do something else in order to restore API functionality?

[RichieB2B]

No,this enables the option to use the new Fleet API. To keep using the old Owner API we need to merge #548 as well.

[ChutneyMary]

My TWCManager has stopped working for API control and I hope someone could provide guidance on where to go from here...
There are two options in the Tesla Developer portal -

Option 1: 'Authorisation Code and Machine-to-Machine'.
Option 2: 'Machine-to-Machine only'.

Option 1: Requests a domain name and callback URL. I've set up a web-server with the public key com.tesla.3p.public-key.pem being available at my domain. I can successfully test retrieve the key. I get variations of 'Unable to onboard app' or 'Unable to fulfil request, try again later' when I try to set up an account. I've done this dozens of times over a few weeks.

Option 2: I appear to have had success using the option 'Machine-to-Machine'. In the Tesla Developer portal, it provides details of OAuth 2.0 Client ID and Client Secret.

What I don't yet understand is whether the OAuth 2.0 Client ID and Client Secret is sufficient to continue? Is the Option 1 method with public key retrieval absolutely required?

[RichieB2B]

@ChutneyMary Here is a nice tutorial that can be used: https://shankar-k.medium.com/tesla-developer-api-guide-account-setup-app-creation-registration-and-third-party-da24aba1bddd

[alexeiw123]

Can someone provide advice on how I can point my TWCManager to a working fleet API that I'm hosting.

I have set up access to fleet API through home assistant and it is now working for the home assistant integration.

I have done a git pull and make install on the main branch but I'm not able to see the following lines in my config.json. I feel that I'm making some basic error...

TWCManager/etc/twcmanager/config.json

Lines 139 to 159 in f4dcb13

	# Since 2024 the Tesla Vehicle Command protocol is required for all
	# vehicles manufacturered after 2021. The Tesla Vehicle Command proxy
	# will automatically fall back to the FleetAPI for older vehicles.
	#
	# TWCManager does not implement the Tesla Vehicle Command protocol.
	# Instead it relies on the Tesla Vehicle Command proxy which translates
	# the FleetAPI REST calls if the destination vehicle supports the
	# new protocol. This requires the URL of the proxy to be set as
	# "teslaProxy" and the proxy certificate for validation as
	# "teslaProxyCert".
	# See https://github.com/teslamotors/vehicle-command
	#
	# The URL of the Tesla HTTP Proxy running locally
	# "teslaProxy": "https://localhost:4443",
	#
	# Only set "teslaProxyCert" when "teslaProxy" is set.
	#"teslaProxyCert": "/path/to/public_key.pem",
	# The cloudUpdateInterval determines how often to poll certain
	# data retrieved from the Tesla API to evaluate policy.
	"cloudUpdateInterval": 1800,