UNFIMP sees nearly all devices as offline

[BenDavidson90]

Is there an existing issue for this?

• I have searched the existing open and closed issues and I checked the docs https://github.com/jokob-sk/NetAlertX/tree/main/docs

Current Behavior

Hi,
I enabled the UNFIMP plugin and connected it to my UDM.
It sees all devices but tag nearly all of them as offline. In result, I only see 5 devices online out of 73 imported.
Here are some screenshots:

Expected Behavior

Display all online devices

Steps To Reproduce

No response

app.conf

No response

docker-compose.yml

No response

What branch are you running?

Production

app.log

app_anon.log

Debug enabled

• I have read and followed the steps in the wiki link above and provided the required debug logs and the log section covers the time when the issue occurs.

[jokob-sk]

Hi,

ARPSCAN might interfere with the results. Can you try to disable it?

Also, have you tried different values fro the UNFIMP_fullimport setting?

[BenDavidson90]

Hi,

ARPSCAN might interfere with the results. Can you try to disable it?

Fresh reinstalled it, ARPSCAN disabled:

Same result. Here is the report of the last scan:

Also, have you tried different values fro the UNFIMP_fullimport setting?

Tried disabled, once and always...

[jokob-sk]

Hi @BenDavidson90 ,

Can you please have a look at the docker logs if the backend is restarting for some reason?

I can see in your logs that after the plugin run the app back end seems to restart, indicated by these log entries:

13:20:02 [Process Scan] Stats end
13:20:02 [Process Scan] Sessions Events (connect / discconnect)
13:20:02 [Process Scan] Creating new devices
13:20:03 [MAIN] Setting up ...
13:20:03 [conf.tz] Setting up ...

The app log doesn't contain any exceptions, so please check the docker logs (e.g. via Portainer) to see if there is an exception logged there.

[jokob-sk]

I think I found the issue.

It would be great if you could test this. Can you please switch to the netalertx-dev docker image (backup everything at first), in about 15 minutes (or after the last action finishes) from now.

Thanks in advance,
j

[BenDavidson90]

Hi @jokob-sk , here are some docker logs that I think can help:

10:00:15 [Plugin utils] ---------------------------------------------
10:00:15 [Plugin utils] display_name: UniFi import
10:00:15 [Plugins] Executing: python3 /app/front/plugins/unifi_import/script.py
10:00:16 [Plugins] SUCCESS, received 68 entries
10:00:19 [API] Updating table_appevents.json file in /front/api
10:00:19 [API] Updating table_plugins_history.json file in /front/api
10:00:19 [API] Updating table_plugins_objects.json file in /front/api
10:00:19 [Plugin utils] ---------------------------------------------
10:00:19 [Plugin utils] display_name: Internet-Check
10:00:19 [Plugins] Executing: python3 /app/front/plugins/internet_ip/script.py prev_ip={prev_ip} INTRNT_DIG_GET_IP_ARG={INTRNT_DIG_GET_IP_ARG}
10:00:20 [Plugins] SUCCESS, received 1 entries
10:00:20 [API] Updating table_appevents.json file in /front/api
10:00:20 [API] Updating table_plugins_history.json file in /front/api
10:00:20 [Plugin utils] ---------------------------------------------
10:00:20 [Plugin utils] display_name: DB cleanup
10:00:20 [Plugins] Executing: python3 /app/front/plugins/db_cleanup/script.py pluginskeephistory={pluginskeephistory} hourstokeepnewdevice={hourstokeepnewdevice} daystokeepevents={daystokeepevents} pholuskeepdays={pholuskeepdays}
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/app/server/__main__.py", line 209, in <module>
    sys.exit(main())       
             ^^^^^^
  File "/app/server/__main__.py", line 139, in main
    process_scan(db)
  File "/app/server/networkscan.py", line 37, in process_scan
    create_new_devices (db)
  File "/app/server/device.py", line 244, in create_new_devices
    '{sanitize_SQL_input(cur_PORT)}',
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/server/helper.py", line 813, in sanitize_SQL_input
    return val.replace("'", "_")
           ^^^^^^^^^^^
AttributeError: 'int' object has no attribute 'replace'
10:00:20 [Plugins] No output received from the plugin DBCLNP - enable LOG_LEVEL=debug and check logs
10:00:20 [Process Scan]  Processing scan results
10:00:20 [Process Scan] Print Stats
10:00:20 [Scan Stats] Devices Detected.......: 70
10:00:20 [Scan Stats] New Devices............: 63
10:00:20 [Scan Stats] Down Alerts............: 0
10:00:20 [Scan Stats] New Down Alerts........: 0
10:00:20 [Scan Stats] New Connections........: 0
10:00:20 [Scan Stats] Disconnections.........: 1
10:00:20 [Scan Stats] IP Changes.............: 0
10:00:20 [Scan Stats] Scan Method Statistics:
10:00:20     INTRNT: 1
10:00:20     UNFIMP: 68
10:00:20     local_MAC: 1
10:00:20 [Process Scan] Stats end
10:00:20 [Process Scan] Sessions Events (connect / discconnect)
10:00:20 [Process Scan] Creating new devices

            [INSTALL] 🚀 Starting app (:20211)

        
10:00:21 [MAIN] Setting up ...
10:00:21 [conf.tz] Setting up ...

[jokob-sk]

Thanks @BenDavidson90 - yes it seems like this is the issue I just fixed. If you can try switching to the netalertx-dev image and see if it is resolved.

[BenDavidson90]

Thanks for your work @jokob-sk , tested it and it works!

I've noticed a small bug: it doesn't import letters with accents (e.g. it writes “cam ra” instead of “caméra”), but this has nothing to do with the bug reported 😁

[jokob-sk]

Thanks - regarding the names, should be fixed as well. It will replace caméra -> camera

You know the drill :)

It would be great if you could test this. Can you please switch to the netalertx-dev docker image (backup everything at first), in about 15 minutes (or after the last action finishes) from now.