Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Onvif port invalid literal for int() with base 10: #92603

Closed
nrsf opened this issue May 5, 2023 · 29 comments · Fixed by #92711
Closed

Onvif port invalid literal for int() with base 10: #92603

nrsf opened this issue May 5, 2023 · 29 comments · Fixed by #92711
Assignees
Labels
integration: onvif problem in device Issue lies within the device, such firmware, software, or user customization/config waiting-for-reply

Comments

@nrsf
Copy link

nrsf commented May 5, 2023

The problem

Error started in Core 2023.5
Onvif cams with non standard ports stoped working with Error:

Logger: homeassistant.config_entries
Source: components/onvif/event.py:682
First occurred: 08:27:22 (4 occurrences)
Last logged: 08:31:01

Error setting up entry C06 - 58:10:8c:3f:d3:84 for onvif
Error setting up entry C10 - 58:10:8c:3f:da:11 for onvif
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpx/_urlparse.py", line 339, in normalize_port
    port_as_int = int(port)
ValueError: invalid literal for int() with base 10: '8106:8106'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 387, in async_setup
    result = await component.async_setup_entry(hass, self)
  File "/usr/src/homeassistant/homeassistant/components/onvif/__init__.py", line 39, in async_setup_entry
    await device.async_setup()
  File "/usr/src/homeassistant/homeassistant/components/onvif/device.py", line 151, in async_setup
    self.capabilities.events = await self.async_start_events()
  File "/usr/src/homeassistant/homeassistant/components/onvif/device.py", line 358, in async_start_events
    return await self.events.async_start(pull_point_support is not False, True)
  File "/usr/src/homeassistant/homeassistant/components/onvif/event.py", line 132, in async_start
    events_via_webhook = try_webhook and await self.webhook_manager.async_start()
  File "/usr/src/homeassistant/homeassistant/components/onvif/event.py", line 635, in async_start
    if not await self._async_start_webhook():
  File "/usr/src/homeassistant/homeassistant/components/onvif/event.py", line 693, in _async_start_webhook
    await self._async_create_webhook_subscription()
  File "/usr/src/homeassistant/homeassistant/components/onvif/event.py", line 682, in _async_create_webhook_subscription
    await self._notification_manager.start()
  File "/usr/local/lib/python3.10/site-packages/onvif/client.py", line 331, in start
    await self._service.SetSynchronizationPoint()
  File "/usr/local/lib/python3.10/site-packages/zeep/proxy.py", line 64, in __call__
    return await self._proxy._binding.send_async(
  File "/usr/local/lib/python3.10/site-packages/zeep/wsdl/bindings/soap.py", line 156, in send_async
    response = await client.transport.post_xml(
  File "/usr/local/lib/python3.10/site-packages/zeep/transports.py", line 235, in post_xml
    response = await self.post(address, message, headers)
  File "/usr/local/lib/python3.10/site-packages/zeep/transports.py", line 220, in post
    response = await self.client.post(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1845, in post
    return await self.request(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1517, in request
    request = self.build_request(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 346, in build_request
    url = self._merge_url(url)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 376, in _merge_url
    merge_url = URL(url)
  File "/usr/local/lib/python3.10/site-packages/httpx/_urls.py", line 113, in __init__
    self._uri_reference = urlparse(url, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/httpx/_urlparse.py", line 246, in urlparse
    parsed_port: typing.Optional[int] = normalize_port(port, scheme)

What version of Home Assistant Core has the issue?

2023.5.1

What was the last working version of Home Assistant Core?

2023.4.4

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Onvif

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Logger: homeassistant.config_entries
Source: components/onvif/event.py:682
First occurred: 08:27:22 (4 occurrences)
Last logged: 08:31:01

Error setting up entry C06 - 58:10:8c:3f:d3:84 for onvif
Error setting up entry C10 - 58:10:8c:3f:da:11 for onvif
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpx/_urlparse.py", line 339, in normalize_port
    port_as_int = int(port)
ValueError: invalid literal for int() with base 10: '8106:8106'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 387, in async_setup
    result = await component.async_setup_entry(hass, self)
  File "/usr/src/homeassistant/homeassistant/components/onvif/__init__.py", line 39, in async_setup_entry
    await device.async_setup()
  File "/usr/src/homeassistant/homeassistant/components/onvif/device.py", line 151, in async_setup
    self.capabilities.events = await self.async_start_events()
  File "/usr/src/homeassistant/homeassistant/components/onvif/device.py", line 358, in async_start_events
    return await self.events.async_start(pull_point_support is not False, True)
  File "/usr/src/homeassistant/homeassistant/components/onvif/event.py", line 132, in async_start
    events_via_webhook = try_webhook and await self.webhook_manager.async_start()
  File "/usr/src/homeassistant/homeassistant/components/onvif/event.py", line 635, in async_start
    if not await self._async_start_webhook():
  File "/usr/src/homeassistant/homeassistant/components/onvif/event.py", line 693, in _async_start_webhook
    await self._async_create_webhook_subscription()
  File "/usr/src/homeassistant/homeassistant/components/onvif/event.py", line 682, in _async_create_webhook_subscription
    await self._notification_manager.start()
  File "/usr/local/lib/python3.10/site-packages/onvif/client.py", line 331, in start
    await self._service.SetSynchronizationPoint()
  File "/usr/local/lib/python3.10/site-packages/zeep/proxy.py", line 64, in __call__
    return await self._proxy._binding.send_async(
  File "/usr/local/lib/python3.10/site-packages/zeep/wsdl/bindings/soap.py", line 156, in send_async
    response = await client.transport.post_xml(
  File "/usr/local/lib/python3.10/site-packages/zeep/transports.py", line 235, in post_xml
    response = await self.post(address, message, headers)
  File "/usr/local/lib/python3.10/site-packages/zeep/transports.py", line 220, in post
    response = await self.client.post(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1845, in post
    return await self.request(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1517, in request
    request = self.build_request(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 346, in build_request
    url = self._merge_url(url)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 376, in _merge_url
    merge_url = URL(url)
  File "/usr/local/lib/python3.10/site-packages/httpx/_urls.py", line 113, in __init__
    self._uri_reference = urlparse(url, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/httpx/_urlparse.py", line 246, in urlparse
    parsed_port: typing.Optional[int] = normalize_port(port, scheme)
  File "/usr/local/lib/python3.10/site-packages/httpx/_urlparse.py", line 341, in normalize_port
    raise InvalidURL("Invalid port")
httpx.InvalidURL: Invalid port

Additional information

No response

@home-assistant
Copy link

home-assistant bot commented May 5, 2023

Hey there @hunterjm, mind taking a look at this issue as it has been labeled with an integration (onvif) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of onvif can trigger bot actions by commenting:

  • @home-assistant close Closes the issue.
  • @home-assistant rename Awesome new title Renames the issue.
  • @home-assistant reopen Reopen the issue.
  • @home-assistant unassign onvif Removes the current integration label and assignees on the issue, add the integration domain after the command.

(message by CodeOwnersMention)


onvif documentation
onvif source
(message by IssueLinks)

@bdraco
Copy link
Member

bdraco commented May 5, 2023

What do you have the port set to?

Please post diagnostics.

@nrsf
Copy link
Author

nrsf commented May 5, 2023

Hello @bdraco !

Port I'am using in the cameras with issues: 8106.

Logger: homeassistant.config_entries
Source: components/onvif/event.py:682
First occurred: 09:25:08 (2 occurrences)
Last logged: 09:25:41

Error setting up entry C06 - 58:10:8c:3f:d3:84 for onvif
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpx/_urlparse.py", line 339, in normalize_port
    port_as_int = int(port)
ValueError: invalid literal for int() with base 10: '8106:8106'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 387, in async_setup
    result = await component.async_setup_entry(hass, self)
  File "/usr/src/homeassistant/homeassistant/components/onvif/__init__.py", line 39, in async_setup_entry
    await device.async_setup()
  File "/usr/src/homeassistant/homeassistant/components/onvif/device.py", line 151, in async_setup
    self.capabilities.events = await self.async_start_events()
  File "/usr/src/homeassistant/homeassistant/components/onvif/device.py", line 358, in async_start_events
    return await self.events.async_start(pull_point_support is not False, True)
  File "/usr/src/homeassistant/homeassistant/components/onvif/event.py", line 132, in async_start
    events_via_webhook = try_webhook and await self.webhook_manager.async_start()
  File "/usr/src/homeassistant/homeassistant/components/onvif/event.py", line 635, in async_start
    if not await self._async_start_webhook():
  File "/usr/src/homeassistant/homeassistant/components/onvif/event.py", line 693, in _async_start_webhook
    await self._async_create_webhook_subscription()
  File "/usr/src/homeassistant/homeassistant/components/onvif/event.py", line 682, in _async_create_webhook_subscription
    await self._notification_manager.start()
  File "/usr/local/lib/python3.10/site-packages/onvif/client.py", line 331, in start
    await self._service.SetSynchronizationPoint()
  File "/usr/local/lib/python3.10/site-packages/zeep/proxy.py", line 64, in __call__
    return await self._proxy._binding.send_async(
  File "/usr/local/lib/python3.10/site-packages/zeep/wsdl/bindings/soap.py", line 156, in send_async
    response = await client.transport.post_xml(
  File "/usr/local/lib/python3.10/site-packages/zeep/transports.py", line 235, in post_xml
    response = await self.post(address, message, headers)
  File "/usr/local/lib/python3.10/site-packages/zeep/transports.py", line 220, in post
    response = await self.client.post(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1845, in post
    return await self.request(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1517, in request
    request = self.build_request(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 346, in build_request
    url = self._merge_url(url)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 376, in _merge_url
    merge_url = URL(url)
  File "/usr/local/lib/python3.10/site-packages/httpx/_urls.py", line 113, in __init__
    self._uri_reference = urlparse(url, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/httpx/_urlparse.py", line 246, in urlparse
    parsed_port: typing.Optional[int] = normalize_port(port, scheme)
  File "/usr/local/lib/python3.10/site-packages/httpx/_urlparse.py", line 341, in normalize_port
    raise InvalidURL("Invalid port")
httpx.InvalidURL: Invalid port

@nrsf
Copy link
Author

nrsf commented May 5, 2023

image

@bdraco
Copy link
Member

bdraco commented May 5, 2023

Please post the json values from /config/.storage/core.config_entries

@nrsf
Copy link
Author

nrsf commented May 5, 2023

  {
        "entry_id": "3eeea663c6e9a12a2df9d6b7671e5bdc",
        "version": 1,
        "domain": "onvif",
        "title": "C06 - 58:10:8c:3f:d3:88",
        "data": {
          "name": "C06",
          "host": "192.168.1.106",
          "port": 8106,
          "username": "uuuuuuuuuuuuu",
          "password": "pppppppppppp"
        },
        "options": {
          "extra_arguments": "-pred 1",
          "rtsp_transport": "tcp"
        },
        "pref_disable_new_entities": false,
        "pref_disable_polling": false,
        "source": "user",
        "unique_id": "58:10:8c:3f:d3:84",
        "disabled_by": null
      }

@bdraco
Copy link
Member

bdraco commented May 5, 2023

Thanks. That looks right.

Will have to dig some more to see how that can happen

@nrsf
Copy link
Author

nrsf commented May 5, 2023

The following one is OK (working). In CORE.2023.4.* all of them worked.

{
        "entry_id": "d8f2f07a435a41a592c10580dd2e1e8a",
        "version": 1,
        "domain": "onvif",
        "title": "GARAGEM - 00:12:31:51:12:cb",
        "data": {
          "name": "GARAGEM",
          "host": "192.168.111.80",
          "port": 8899,
          "username": "uuuuuuuuuu",
          "password": "pppppppppp",
          "snapshot_auth": "digest"
        },
        "options": {
          "extra_arguments": "-pred 1",
          "rtsp_transport": "tcp"
        },
        "pref_disable_new_entities": false,
        "pref_disable_polling": false,
        "source": "user",
        "unique_id": "00:12:31:51:12:cb",
        "disabled_by": null
      }

@bdraco
Copy link
Member

bdraco commented May 5, 2023

So I think what is happening is that the camera is sending back an invalid URL with the port in the url twice in the xaddr list for the new webhook support that was added in 2023.5.x. You likely didn't hit the bug in the camera firmware before because we never tried to enable notifications before.

Can you enable debug logs for onvif homeassistant.components.onvif and zeep (https://www.home-assistant.io/integrations/logger/)

# Example configuration.yaml entry
logger:
  default: info
  logs:
    homeassistant.components.onvif: debug
    onvif: debug
    zeep: debug

Restart HA

Reload the config entry and than post the log

@bdraco
Copy link
Member

bdraco commented May 5, 2023

Also please post a link to the model information for the camera and where you purchased it

@bdraco bdraco added waiting-for-reply problem in device Issue lies within the device, such firmware, software, or user customization/config labels May 5, 2023
@nrsf
Copy link
Author

nrsf commented May 6, 2023

So I think what is happening is that the camera is sending back an invalid URL with the port in the url twice in the xaddr list for the new webhook support that was added in 2023.5.x. You likely didn't hit the bug in the camera firmware before because we never tried to enable notifications before.

Can you enable debug logs for onvif homeassistant.components.onvif and zeep (https://www.home-assistant.io/integrations/logger/)

# Example configuration.yaml entry
logger:
  default: info
  logs:
    homeassistant.components.onvif: debug
    onvif: debug
    zeep: debug

Restart HA

Reload the config entry and than post the log

I followed the instructions to enable debug logs. After restarting HA with debug logs turned on I was unable to get the log file when reloading an ONVIF camera that had a problem.

However when I reloaded a working camera I managed to get the log file which I think is useless since said camera was OK.

@nrsf
Copy link
Author

nrsf commented May 6, 2023

Also please post a link to the model information for the camera and where you purchased it

Camera info:
image

Manufacturer: Intelbras (Brazil)
Model: VIP-S3020-G2

Link: https://www.intelbras.com/pt-br/ajuda-download/download/camera-bullet-ip-vip-s3020-g2

@nrsf
Copy link
Author

nrsf commented May 6, 2023

" You likely didn't hit the bug in the camera firmware before because we never tried to enable notifications before."

Any chance the "enable notifications" to be optional to the ONVIF integration? If that is the probleam making it optional would make non well behaving cameras that was working before work again...

Thank you very much for all the help!

@bdraco
Copy link
Member

bdraco commented May 6, 2023

We generally don’t want to add a matrix of check boxes to deal with broken camera implementations since it shifts the burden to the user to figure on which combination of check boxes they need to check when they set up the integration.

It would be better if you can get me logs so we can work around the bug in the camera instead

@bdraco
Copy link
Member

bdraco commented May 6, 2023

Navigate to https://YOUR_INSTANCE:8123/config/logs

Scroll down to the bottom

Click Download Full Log

@nrsf
Copy link
Author

nrsf commented May 6, 2023

@nrsf
Copy link
Author

nrsf commented May 6, 2023

You are right...
<wsa5:Address>http://192.168.1.106:8106:8106/onvif/Subscription?Idx=43</wsa5:Address>

@bdraco
Copy link
Member

bdraco commented May 6, 2023

>>> from urllib.parse import urlparse
>>> urlparse("http://192.168.1.106:8106:8106/onvif/Subscription?Idx=43")
ParseResult(scheme='http', netloc='192.168.1.106:8106:8106', path='/onvif/Subscription', params='', query='Idx=43', fragment='')
if "[" not in url.netloc and url.netloc.count(":") > 1

I think we can fixup the urls but I have to get the urls extracted first which means we need to wait for openvideolibs/python-onvif-zeep-async#32

@bdraco
Copy link
Member

bdraco commented May 7, 2023

@nrsf Can you try the fix?

cd /config
curl -o- -sSL https://gist.githubusercontent.com/bdraco/43f8043cb04b9838383fd71353e99b18/raw/core_integration_pr | bash /dev/stdin -d onvif -p 92711
cat /config/custom_components/onvif/manifest.json

Verify that the version displayed above is "onvif-zeep-async==3.1.0" and NOT 2.1.1

Restart HA

Test

Delete /config/custom_components/onvif and restart after testing is completed to restore the core version

@nrsf
Copy link
Author

nrsf commented May 7, 2023

[core-ssh config]$ cat /config/custom_components/onvif/manifest.json { "domain": "onvif", "name": "ONVIF", "codeowners": [ "@hunterjm" ], "config_flow": true, "dependencies": [ "ffmpeg" ], "dhcp": [ { "registered_devices": true } ], "documentation": "https://www.home-assistant.io/integrations/onvif", "iot_class": "local_push", "loggers": [ "onvif", "wsdiscovery", "zeep" ], "requirements": [ "onvif-zeep-async==3.1.0", "WSDiscovery==2.0.0" ], "version": "92711" }

@nrsf
Copy link
Author

nrsf commented May 7, 2023

Test after applying the fix and restarting:
`Este erro originou-se de uma integração personalizada.

Logger: homeassistant.config_entries
Source: custom_components/onvif/init.py:123
Integration: ONVIF
First occurred: 07:54:54 (1 occurrences)
Last logged: 07:54:54

Error setting up entry C06 - 58:10:8c:3f:d3:84 for onvif
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/httpx/_urlparse.py", line 339, in normalize_port
port_as_int = int(port)
ValueError: invalid literal for int() with base 10: '8106:8106'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/config_entries.py", line 387, in async_setup
result = await component.async_setup_entry(hass, self)
File "/config/custom_components/onvif/init.py", line 42, in async_setup_entry
await async_populate_snapshot_auth(hass, device, entry)
File "/config/custom_components/onvif/init.py", line 136, in async_populate_snapshot_auth
auth = await _get_snapshot_auth(device)
File "/config/custom_components/onvif/init.py", line 123, in _get_snapshot_auth
snapshot = await device.device.get_snapshot(device.profiles[0].token)
File "/usr/local/lib/python3.10/site-packages/onvif/client.py", line 496, in get_snapshot
response = await self._snapshot_client.get(uri, auth=auth)
File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1754, in get
return await self.request(
File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1517, in request
request = self.build_request(
File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 346, in build_request
url = self._merge_url(url)
File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 376, in _merge_url
merge_url = URL(url)
File "/usr/local/lib/python3.10/site-packages/httpx/_urls.py", line 113, in init
self._uri_reference = urlparse(url, **kwargs)
File "/usr/local/lib/python3.10/site-packages/httpx/_urlparse.py", line 246, in urlparse
parsed_port: typing.Optional[int] = normalize_port(port, scheme)
File "/usr/local/lib/python3.10/site-packages/httpx/_urlparse.py", line 341, in normalize_port
raise InvalidURL("Invalid port")
httpx.InvalidURL: Invalid port
`

@bdraco
Copy link
Member

bdraco commented May 7, 2023

So it’s getting further. That’s good. Will need some more adjustments

bdraco added a commit to openvideolibs/python-onvif-zeep-async that referenced this issue May 7, 2023
@bdraco
Copy link
Member

bdraco commented May 7, 2023

@nrsf Can you try the updated fix with onvif-zeep-async==3.1.1? (It may take another iteration or two as I'm working around each new error as you give me new logs)

cd /config
curl -o- -sSL https://gist.githubusercontent.com/bdraco/43f8043cb04b9838383fd71353e99b18/raw/core_integration_pr | bash /dev/stdin -d onvif -p 92711
cat /config/custom_components/onvif/manifest.json

Verify that the version displayed above is "onvif-zeep-async==3.1.1" and NOT 2.1.1

Restart HA

Test

Delete /config/custom_components/onvif and restart after testing is completed to restore the core version

@nrsf
Copy link
Author

nrsf commented May 7, 2023

{ "domain": "onvif", "name": "ONVIF", "codeowners": [ "@hunterjm" ], "config_flow": true, "dependencies": [ "ffmpeg" ], "dhcp": [ { "registered_devices": true } ], "documentation": "https://www.home-assistant.io/integrations/onvif", "iot_class": "local_push", "loggers": [ "onvif", "wsdiscovery", "zeep" ], "requirements": [ "onvif-zeep-async==3.1.1", "WSDiscovery==2.0.0" ], "version": "92711" }

@nrsf
Copy link
Author

nrsf commented May 7, 2023

Thank you very, very much!

You have nailed it!
It is working now!!
Could it be a problem I keep using custom (fixed) version until the fix get released? If no problem: Should I remove the custom version before applying the next core update?

@bdraco
Copy link
Member

bdraco commented May 7, 2023

I'll try to get the change in 2023.6.x or sooner depending on how busy my day job is next week

@bdraco
Copy link
Member

bdraco commented May 14, 2023

Please try 2023.5.4 when it is released

@nrsf
Copy link
Author

nrsf commented Jun 8, 2023

Hello @bdraco
Thank you very much!!!
Just upgraded to 2023.6.0 and everything is working fine!

@bdraco
Copy link
Member

bdraco commented Jun 8, 2023

Thanks for confirming 👍

@github-actions github-actions bot locked and limited conversation to collaborators Jul 8, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
integration: onvif problem in device Issue lies within the device, such firmware, software, or user customization/config waiting-for-reply
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants