[BUG] Mycroft Mark II (user-facing apparent) forgets WiFi after restart or power cycle (red LED)

[fermulator]

Description

this appears to be a non-functional issue, the WiFi is still configured and functional, however in the UI, and the RED LED on top of Mark II - it thinks is NOT configured

went through a fresh install https://neon.ai/NeonAIforMycroftMarkII/ today

configured only basics

1. WiFi
2. home-assistant
3. change to "metric" units

home-assistant wasn't actually fully working, so I tried these update steps
https://neongeckocom.github.io/neon-docs/neon_os/neon_os_updates/#manual-update-via-terminal

issued a full restart -

then/now the UI shows unconfigured WiFi and red LED on top is illuminated

Steps to Reproduce

1. install fresh on Mark II
2. configure basics (wifi, etc)
3. (possibly run the manual update steps)
4. issue restart

Relevant Code

No response

Other Notes

same as MycroftAI/mycroft-dinkum#52

[NeonDaniel]

If you select the previously-connected network, does it remember your input credentials, or do you have to re-enter them?

[fermulator]

more troubleshooting details in here

https://matrix.to/#/!ZhEZYNzKBfpEAhtQIz:matrix.org/$Fzt1jkdxXUUQ2oXPCUcnq86YDp0dy6xmkQh3YFuXX9I?via=matrix.org&via=nitro.chat&via=strugee.net

basically - in my network, I dole out DNS via DHCP - the MarkII is (appropriately) on an IoT isolated VLAN wifi network, AND it is subject to firewall rules, namely BLOCK DNS egress (unless it goes to the DNS server I instructed or the pihole ...

it seems that our "test connection" code isn't honoring the configured DNS on the system

$ ping google.ca
PING google.ca (142.251.33.163) 56(84) bytes of data.
64 bytes from yyz10s17-in-f3.1e100.net (142.251.33.163): icmp_seq=1 ttl=115 time=7.51 ms
^C

$ cat /etc/resolv.conf
# (snip)

nameserver 127.0.0.53
options edns0 trust-ad
search .

$ resolvectl
Global
       Protocols: +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: foreign
       DNS Domain ~.

Link 2 (end0)
Current Scopes: none
     Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported

Link 3 (wlan0)
    Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
         Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 10.0.0.60
       DNS Servers: 10.0.0.60
        DNS Domain: local

and:

$ python -i ./venv/lib/python3.10/site-packages/ovos_utils/network_utils.py

>>> get_ip()
'192.168.12.83'
>>> get_network_tests_config()
{'ip_url': 'https://api.ipify.org', 'dns_primary': '1.1.1.1', 'dns_secondary': '8.8.8.8', 'web_url': 'http://nmcheck.gnome.org/check_network_status.txt', 'web_url_secondary': 'https://checkonline.home-assistant.io/online.txt', 'captive_portal_url': 'http://nmcheck.gnome.org/check_network_status.txt', 'captive_portal_text': 'NetworkManager is online'}
>>> is_connected_dns()
False
>>> is_connected_http()
True
>>> 

[fermulator]

https://github.com/OpenVoiceOS/ovos-utils/blob/master/ovos_utils/network_utils.py

(default test config 1.1.1.1 or 8.8.8.8)

[fermulator]

here's a potential code fix for directional change ..

import socket
import subprocess
from typing import Optional, List

_DEFAULT_TEST_CONFIG = {
    'dns_primary': '1.1.1.1',
    'dns_secondary': '8.8.8.8'
}

def get_system_dns_servers() -> List[str]:
    """
    Retrieve the DNS servers from the system's configuration.
    Returns:
        A list of DNS server addresses.
    """
    try:
        # This command retrieves the DNS servers configured by systemd-resolved
        output = subprocess.check_output(['systemd-resolve', '--status'], text=True)
        dns_servers = []
        for line in output.splitlines():
            if 'DNS Servers' in line:
                # The line below DNS Servers should contain the actual addresses
                dns_servers = line.split(':')[1].strip().split()
        return dns_servers
    except Exception as e:
        print(f"Error retrieving DNS servers: {e}")
        return [_DEFAULT_TEST_CONFIG['dns_primary'], _DEFAULT_TEST_CONFIG['dns_secondary']]

def is_connected_dns(host: Optional[str] = None, port: int = 53,
                     timeout: int = 3) -> bool:
    """
    Check internet connection by connecting to DNS servers.
    Returns:
        True if internet connection can be detected.
    """

    if host is None:
        # Get the system's DNS servers
        dns_servers = get_system_dns_servers()
        
        # Attempt to connect to the DNS servers
        for dns in dns_servers:
            if is_connected_dns(dns, port, timeout):
                return True

    try:
        # Connect to the host -- tells us if the host is actually reachable
        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        s.settimeout(timeout)
        s.connect((host, port))
        return True
    except OSError:
        pass
    return False

(chat GPT generated)

The logic for how we want to assert connection is basically a "first success = true" (starting with system DNS) ...

it's a bit tricky to get system DNS from python code ... OR we use this library

import socket
import dns.resolver
from typing import Optional

_DEFAULT_TEST_CONFIG = {
    'dns_primary': '1.1.1.1',
    'dns_secondary': '8.8.8.8'
}

def get_system_dns_servers() -> list:
    """
    Retrieve the DNS servers used by the system using dnspython.
    Returns:
        A list of DNS server addresses.
    """
    resolver = dns.resolver.Resolver()
    return resolver.nameservers

def is_connected_dns(host: Optional[str] = None, timeout: int = 3) -> bool:
    """
    Check internet connection by connecting to DNS servers.
    Returns:
        True if internet connection can be detected.
    """
    dns_servers = get_system_dns_servers()

    if host is None:
        # Test connectivity with DNS resolution
        for dns in dns_servers:
            try:
                # Attempt to resolve a known domain
                dns.resolver.Resolver(configure=False).nameservers = [dns]
                dns.resolver.Resolver(configure=False).resolve('google.com', 'A')
                return True
            except (dns.resolver.NoAnswer, dns.resolver.NXDOMAIN, dns.resolver.Timeout):
                continue  # Try the next DNS server

    try:
        # Connect to the host to check connectivity
        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        s.settimeout(timeout)
        s.connect((host, 53))
        return True
    except OSError:
        pass
    return False

[NeonDaniel]

Thanks for the troubleshooting and detailed proposed solution! The relevant Matrix chat also has some context for this.

My suggestion for a solution would be to update ovos-utils to use dns.resolver to determine the system DNS as suggested in the second example above.

The _DEFAULT_TEST_CONFIG could remove default dns addresses and then treat a configured value as an override with default behavior being to ask the system what DNS to test.

[fermulator]

dropped in new ticket here per request OpenVoiceOS/ovos-utils#272 - since we could not move/transfer issue
(we can close this one - troubleshooting complete ;o)