Issue with VPNUser data in 'wrong' format. #51

deltaphi · 2022-09-16T18:20:39Z

I'm using a Fritz!Box 7520 with Fritz!OS 7.29. I've tried with fritzinfluxdb master and next-release (as of the time of writing), both show the same error.

Fritzinfluxdb can connect to the FritzBox:

fritzinfluxdb  | 2022-09-16 18:14:26,803 - INFO: Successfully established FritzBox Lua session
fritzinfluxdb  | 2022-09-16 18:14:26,804 - ERROR: Initializing connection to InfluxDB failed
fritzinfluxdb exited with code 1
fritzinfluxdb  | 2022-09-16 18:14:27,911 - INFO: Starting fritzinfluxdb v1.0.1 (2022-08-22)
fritzinfluxdb  | 2022-09-16 18:14:27,939 - INFO: Successfully parsed config
fritzinfluxdb  | 2022-09-16 18:14:27,946 - INFO: Connection to InfluxDB v2.4.0 established and bucket is present
fritzinfluxdb  | 2022-09-16 18:14:29,471 - INFO: Successfully established FritzBox TR-069 session
fritzinfluxdb  | 2022-09-16 18:14:29,853 - INFO: Successfully established FritzBox Lua session
fritzinfluxdb  | 2022-09-16 18:14:29,853 - INFO: Successfully connected to FritzBox '192.168.178.1' (fritz.box) Model: FRITZ!Box 7520 (UI) - FW: 7.29
fritzinfluxdb  | 2022-09-16 18:14:29,854 - INFO: Starting main loop

But it cannot read any data (REDACTED added by me):

fritzinfluxdb  | 2022-09-16 18:14:30,529 - INFO: Requested invalid action 'GetInfo' for service: WANIPConnection:1
fritzinfluxdb  | 2022-09-16 18:14:30,529 - INFO: Querying action 'GetInfo' will be disabled
fritzinfluxdb  | 2022-09-16 18:14:30,529 - INFO: All actions for service 'WANIPConnection:1' are unavailable. Disabling service.
fritzinfluxdb  | 2022-09-16 18:14:41,651 - ERROR: FritzBox metric type '<class 'dict'>' does not match data: <class 'list'>
fritzinfluxdb  | 2022-09-16 18:14:41,651 - ERROR: FritzBox metric type '<class 'dict'>' does not match data: <class 'list'>
fritzinfluxdb  | 2022-09-16 18:14:41,651 - ERROR: FritzBox metric type '<class 'dict'>' does not match data: <class 'list'>
fritzinfluxdb  | 2022-09-16 18:14:41,651 - ERROR: FritzBox metric type '<class 'dict'>' does not match data: <class 'list'>
fritzinfluxdb  | 2022-09-16 18:14:41,652 - ERROR: Unable to extract 'None' form '{'pid': 'shareVpn', 'hide': {'faxSet': True, 'shareUsb': True, 'ssoSet': True, 'liveTv': True}, 'time': [], 'data': {'vpnInfo': {'userConnections': [], 'type': 'IPSec Xauth PSK', 'help': {'url': 'https://www.avm.de/service/vpn', 'text': 'www.avm.de/service/vpn'}, 'boxConnections': [], 'server': 'REDACTED.myfritz.net'}}, 'sid': 'REDACTED'}', got '<class 'NoneType'>'

Everything starting from FritzBox metric type '<class 'dict'>' does not match data: <class 'list'> will repeat in the log indefinitely.

I found the same problem mentioned in #42, but I'm not using a beta firmware and using next-release didn't do anything for me, so I'm stumped.

What do I need to do to fix getting data from my Fritz!Box?

The text was updated successfully, but these errors were encountered:

bb-Ricardo · 2022-09-16T18:28:05Z

Hi,

Can you run it with -v option so we can see where the error occurs? Thank you

deltaphi · 2022-09-16T18:33:30Z

Hi @bb-Ricardo,

I'm using your docker-compose file to run fritzinfluxdb (and influxdb and grafana). I'm not aware how I would pass a parameter to a command inside a docker container from docker compose. Is there an environment varialbe I can use ot increase the verbosity?

That also said, I just realized: I'm using neither master nor next-release, I'm actually using whatever you have in the docker container bbricardo/fritzinfluxdb:latest.

deltaphi · 2022-09-16T18:48:20Z

After some more thinking: I patched the included Dockerfile to add "-v" to the CMD, built a new containers, and switched over the compose file to use the new container. Now I truly am using master. The error message has now changed - but I'm not sure what to make of this.

First, I'm seeing this line:

fritzinfluxdb  | 2022-09-16 18:38:34,505 - DEBUG: Request FritzBox Lua service 'DSL Info' returned successfully

So it seems to be picking up my internet connection successfully. This got me thinking about the original error: Does WANIPconnection relate to the feature of the Fritz!Box acting as an IP router on an ethernet connection, rather than "any WAN connection"? As I'm on DSL, that would be a plausible explanation for the original log message.

On the other hand, the original error message is now gone. When I search for WANIP in the now fairly extensive logs, I see the following:

fritzinfluxdb  | 2022-09-16 18:38:21,890 - DEBUG: Request FritzBox TR-069 service 'WANIPConn' returned successfully:
GetStatusInfo ({})

I'm only seeing the original error message in relation to VPN connections now:

fritzinfluxdb  | 2022-09-16 18:38:33,546 - DEBUG: Request FritzBox Lua service 'VPN Users' returned successfully     
fritzinfluxdb  | 2022-09-16 18:38:33,547 - ERROR: FritzBox metric type '<class 'dict'>' does not match data: <class '
list'>                                                                                                               
fritzinfluxdb  | 2022-09-16 18:38:33,547 - ERROR: FritzBox metric type '<class 'dict'>' does not match data: <class '
list'>                                                                                                               
fritzinfluxdb  | 2022-09-16 18:38:33,548 - ERROR: FritzBox metric type '<class 'dict'>' does not match data: <class '
list'>                                                                                                               
fritzinfluxdb  | 2022-09-16 18:38:33,548 - ERROR: FritzBox metric type '<class 'dict'>' does not match data: <class '
list'>                                                                                                               
fritzinfluxdb  | 2022-09-16 18:38:33,549 - ERROR: Unable to extract 'None' form '{'pid': 'shareVpn', 'hide': {'faxSet
': True, 'shareUsb': True, 'ssoSet': True, 'liveTv': True}, 'time': [], 'data': {'vpnInfo': {'userConnections': [], 'type': 'IPSec Xauth PSK', 'help': {'url': 'https://www.avm.de/service/vpn', 'text': 'www.avm.de/service/vpn'}, 'boxCo
nnections': [], 'server': 'REDACTED.myfritz.net'}}, 'sid': 'REDACTED'}', got '<class 'NoneType'>'
fritzinfluxdb  | 2022-09-16 18:38:33,549 - DEBUG: FritzOS version 7.29 not in list of supported versions for VPN User
s: ['7.39']

EDIT: I removed the "-v" on my patched docker container and am now back to the original error message. I'm confused.

bb-Ricardo · 2022-09-16T18:54:23Z

Hi @deltaphi,

docker tag latest is fine.

have to investigate a bit more:

{
    "pid": "shareVpn",
    "hide": {
        "faxSet": true,
        "shareUsb": true,
        "ssoSet": true,
        "liveTv": true
    },
    "time": [],
    "data": {
        "vpnInfo": {
            "userConnections": [],
            "type": "IPSec Xauth PSK",
            "help": {
                "url": "https://www.avm.de/service/vpn",
                "text": "www.avm.de/service/vpn"
            },
            "boxConnections": [],
            "server": "REDACTED.myfritz.net"
        }
    },
    "sid": "REDACTED"
}

This is what my FritzBox returns:

{
    "pid": "shareVpn",
    "hide": {
        "shareUsb": true,
        "liveTv": true,
        "dectRdio": true,
        "rss": true,
        "ssoSet": true,
        "dectMail": true,
        "provServ": true,
        "mobile": true,
        "liveImg": true
    },
    "time": [],
    "data": {
        "vpnInfo": {
            "userConnections": {
                "connection1": {
                    "psk": "XXX",
                    "connected": false,
                    "deletable": true,
                    "active": true,
                    "address": "",
                    "virtualAddress": "XXX",
                    "userId": "XXX",
                    "name": "XXX"
                },
                "connection0": {
                    "psk": "",
                    "connected": false,
                    "deletable": true,
                    "active": true,
                    "address": "",
                    "virtualAddress": "XXX",
                    "userId": "",
                    "name": "XXX"
                }
            },
            "type": "IPSec Xauth PSK",
            "help": {
                "url": "https://www.avm.de/service/vpn",
                "text": "www.avm.de/service/vpn"
            },
            "boxConnections": [],
            "server": "XXX.myfritz.net"
        }
    },
    "sid": "XXX"
}

Where .data.vpnInfo.userConnections is a dictionary and not a list. And I'm running 7.29 as well.

Can you (for testing purposes) add a vpn connection and run it again?

try to add this to your docker compose service.

command:
  - python
  - /app/fritzinfluxdb.py
  - -v

deltaphi · 2022-09-16T19:01:38Z

I actually had a VPN connection set up ages ago on a different box - probably before the times when Fritz!Boxes has actual user management. I have not used it in years and it does not show up in the UI - but I have carried over the configuration (export/import) through several boxes. So maybe this is the cause of this strange VPN entry?

I will have to try setting up a new VPN connection over the weekend. Fritz security requires me to dial on an attached phone, which I unfortunately won't be able to dig out tonight.

bb-Ricardo · 2022-09-16T19:21:28Z

Ahh, that is indeed interesting. Maybe it is caused by an old config.

I will try to add a fix, tonight. I mean that can always happen to other users as well.

bb-Ricardo · 2022-09-16T19:22:19Z

And when FritzBox request an physical approval I was always able to just press any button on the FritzBox.

bb-Ricardo · 2022-09-16T20:14:40Z

Hey, I just pushed a new commit to next-release to fix this issue. You will need to build the consenter local again.

Thank you

deltaphi · 2022-09-17T18:38:23Z

With the latest changes I'm now seeing the following:

fritzinfluxdb  | 2022-09-17 18:35:42,639 - DEBUG: Request FritzBox Lua service 'VPN Users' returned successfully
fritzinfluxdb  | 2022-09-17 18:35:42,639 - ERROR: FritzBox metric type '<class 'dict'>' does not match data: <class 'list'>
fritzinfluxdb  | 2022-09-17 18:35:42,640 - ERROR: FritzBox metric type '<class 'dict'>' does not match data: <class 'list'>
fritzinfluxdb  | 2022-09-17 18:35:42,640 - ERROR: FritzBox metric type '<class 'dict'>' does not match data: <class 'list'>
fritzinfluxdb  | 2022-09-17 18:35:42,640 - ERROR: FritzBox metric type '<class 'dict'>' does not match data: <class 'list'>
fritzinfluxdb  | 2022-09-17 18:35:42,641 - DEBUG: 2022-09-17 18:35:42.639385+00:00: myfritz_host_name=REDACTED.myfritz.net ({'box': 'fritz.box'})
fritzinfluxdb  | 2022-09-17 18:35:42,641 - DEBUG: 2022-09-17 18:35:42.639884+00:00: vpn_type=IPSec Xauth PSK ({'box': 'fritz.box'})
fritzinfluxdb  | 2022-09-17 18:35:42,655 - DEBUG: Successfully wrote 2 measurements to InfluxDB

So from the logs, it would appear that despite the error messages, the lingering VPN connection is picked up. I am unsure whether this solves my initial issue (no data available) - I might now be facing some kind of spurious error with connecting Grafana and InfluxDB and am looking into that.

deltaphi · 2022-09-17T19:13:55Z

Continuing the other error in #54.

bb-Ricardo · 2022-09-17T19:26:55Z

Strange that the error is still there. Was sure it is fixed in next-release.

bb-Ricardo · 2022-09-17T19:33:17Z

The docker images are not updated. You have to do a docker build yourself if you need an image with the latest code

deltaphi · 2022-09-17T19:37:15Z

I did rebuild the docker image, also adding the "-v" flag - which is what gave the above log entries.

bb-Ricardo · 2022-09-17T20:44:12Z

Hi, I just pushed another change to next-release may this fixes the issue.

deltaphi · 2022-09-18T18:09:39Z

Hi @bb-Ricardo,
indeed, the latest next-release does not show any "metric type '<class 'dict'>' does not match" messages anymore.

bb-Ricardo added a commit that referenced this issue Sep 16, 2022

fixes issue with 'wrong' VPN user format #51

92207d9

bb-Ricardo added this to the 1.1 milestone Sep 17, 2022

bb-Ricardo changed the title ~~Cannot receive Data from FritzBox~~ Issue with VPNUser data in 'wrong' format. Sep 17, 2022

deltaphi mentioned this issue Sep 17, 2022

Grafana: "InfluxDB Error: retention policy not found: autogen" #54

Closed

deltaphi closed this as completed Sep 17, 2022

bb-Ricardo added a commit that referenced this issue Sep 17, 2022

fixes issue with vpn user data in wrong format #51

a764f8d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue with VPNUser data in 'wrong' format. #51

Issue with VPNUser data in 'wrong' format. #51

deltaphi commented Sep 16, 2022 •

edited

Loading

bb-Ricardo commented Sep 16, 2022

deltaphi commented Sep 16, 2022 •

edited

Loading

deltaphi commented Sep 16, 2022 •

edited

Loading

bb-Ricardo commented Sep 16, 2022 •

edited

Loading

deltaphi commented Sep 16, 2022

bb-Ricardo commented Sep 16, 2022

bb-Ricardo commented Sep 16, 2022

bb-Ricardo commented Sep 16, 2022

deltaphi commented Sep 17, 2022 •

edited

Loading

deltaphi commented Sep 17, 2022

bb-Ricardo commented Sep 17, 2022

bb-Ricardo commented Sep 17, 2022

deltaphi commented Sep 17, 2022

bb-Ricardo commented Sep 17, 2022

deltaphi commented Sep 18, 2022

Issue with VPNUser data in 'wrong' format. #51

Issue with VPNUser data in 'wrong' format. #51

Comments

deltaphi commented Sep 16, 2022 • edited Loading

bb-Ricardo commented Sep 16, 2022

deltaphi commented Sep 16, 2022 • edited Loading

deltaphi commented Sep 16, 2022 • edited Loading

bb-Ricardo commented Sep 16, 2022 • edited Loading

deltaphi commented Sep 16, 2022

bb-Ricardo commented Sep 16, 2022

bb-Ricardo commented Sep 16, 2022

bb-Ricardo commented Sep 16, 2022

deltaphi commented Sep 17, 2022 • edited Loading

deltaphi commented Sep 17, 2022

bb-Ricardo commented Sep 17, 2022

bb-Ricardo commented Sep 17, 2022

deltaphi commented Sep 17, 2022

bb-Ricardo commented Sep 17, 2022

deltaphi commented Sep 18, 2022

deltaphi commented Sep 16, 2022 •

edited

Loading

deltaphi commented Sep 16, 2022 •

edited

Loading

deltaphi commented Sep 16, 2022 •

edited

Loading

bb-Ricardo commented Sep 16, 2022 •

edited

Loading

deltaphi commented Sep 17, 2022 •

edited

Loading