Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tailscale fails to start on HaOS #440

Open
aaronjamt opened this issue Dec 25, 2024 · 10 comments
Open

Tailscale fails to start on HaOS #440

aaronjamt opened this issue Dec 25, 2024 · 10 comments

Comments

@aaronjamt
Copy link

Problem/Motivation

The addon fails to start after updating to 0.24.0, even after reinstalling the addon and removing existing configuration.

Expected behavior

The addon would start normally after the update

Actual behavior

The addon never starts, and outputs the following logs several times:

/run/s6/basedir/scripts/rc.init: warning: hook /etc/s6-overlay/scripts/stage2_hook.sh exited 135
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service base-addon-banner: starting
s6-rc: warning: unable to start service base-addon-banner: command exited 128
/run/s6/basedir/scripts/rc.init: warning: s6-rc failed to properly bring all the services up! Check your logs (in /run/uncaught-logs/current if you have in-container logging) for more information.
/run/s6/basedir/scripts/rc.init: fatal: stopping the container.
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped

Steps to reproduce

Not sure, update to the latest version? I don't see any other reports of this issue so I doubt it's as simple to recreate as "just" updating.

Proposed changes

N/A

@lmagyar
Copy link
Contributor

lmagyar commented Dec 28, 2024

Really strange, the most basic add-on startup functions fail, not even Tailscale related. Seems that something is really broken on your system. I suggest to:

  • check that supervisor works correctly
  • restore the previous version (from automatic backup created when this update happened), and try again the update

@aaronjamt
Copy link
Author

aaronjamt commented Dec 29, 2024

Really strange, the most basic add-on startup functions fail, not even Tailscale related. Seems that something is really broken on your system. I suggest to:

* check that supervisor works correctly

* restore the previous version (from automatic backup created when this update happened), and try again the update

Supervisor appears to be working fine and I'm currently waiting for the backup to restore. I've been playing around with the actual Docker container and I dropped to a shell, but even trying to run basic commands like cat or ls result in a Bus Error (Core Dumped). I just removed the image and re-pulled it from the HaOS root shell, hoping that would help, but it didn't make a difference.

Update: I just rolled back to a backup (version 0.23.2) and the add-on started properly. I'm reattempting the update now.

@aaronjamt
Copy link
Author

Just updated again and am encountering the same issue. It seems updating from v0.23.2 to v0.24.0 breaks the addon entirely. I've also already erased all container configuration, so I'm not sure what else to try from here.

@lmagyar
Copy link
Contributor

lmagyar commented Dec 29, 2024

At the moment 13778 people reported they use the 0.24.0 version, so it really seems that this is not related to this add-on, more like some docker configuration issue, I really have no better guess. If you manually manage docker images/containers, strange things can happen.

@aaronjamt
Copy link
Author

At the moment 13778 people reported they use the 0.24.0 version, so it really seems that this is not related to this add-on, more like some docker configuration issue, I really have no better guess. If you manually manage docker images/containers, strange things can happen.

Gotcha. Until I started having this issue I haven't done anything manually (in fact, I hadn't even set up SSH access until after this update). I'm not sure what else to try, other than maybe updating to 1.23.3 instead, then to 1.24.0, since I skipped that update initially? The HA CLI utility doesn't allow specifying a version to up/down-grade, so that would have to be entirely manual, which I'd rather avoid if possible.

@lmagyar
Copy link
Contributor

lmagyar commented Dec 29, 2024

The only thing that can fail in those scripts (https://github.com/hassio-addons/addon-base/blob/main/base/s6-overlay/etc/s6-rc/scripts/base-addon-banner and https://github.com/hassio-addons/addon-tailscale/blob/main/tailscale/rootfs/etc/s6-overlay/scripts/stage2_hook.sh) are supervisor api related.

Maybe try to uninstall the add-on, remove any add-on related image from docker (if there remains any at all), remove device on TS management web page, and perform a clean add-on install. You can backup (copy/paste) the config yaml from the add-on UI before uninstall, and add it back after install, and only a TS authentication will be necessary on the add-on's web UI. If this also fails, I'm out of ideas, I can't think anything else then supervisor/docker related things.

@aaronjamt
Copy link
Author

I uninstalled the add-on from the HA web UI (and enabled the toggle to remove all add-on data), verified there were no Tailscale-related images in Docker (docker image ls | grep tail), ran docker system prune to clear out any build data or unused images for good measure, then rebooted the entire Pi. I also removed the node from the main Tailscale dashboard. Once the Pi booted, I installed the Tailscale add-on through the web UI (no manual stuff from the shell or anything like that), let it download and install, and checked the logs to find the exact same error. I'm just going to roll back to v0.23.2 for now since I at least know it works, but if you have any other suggestions I'd be happy to try them.

@lmagyar
Copy link
Contributor

lmagyar commented Dec 29, 2024

Could you please share: Settings / System / Repairs / ... / System Information / Copy

@aaronjamt
Copy link
Author

System Information

version core-2024.12.5
installation_type Home Assistant OS
dev false
hassio true
docker true
user root
virtualenv false
python_version 3.13.0
os_name Linux
os_version 6.6.62-haos-raspi
arch aarch64
timezone America/Phoenix
config_dir /config
Home Assistant Community Store
GitHub API ok
GitHub Content ok
GitHub Web ok
HACS Data ok
GitHub API Calls Remaining 4999
Installed Version 2.0.1
Stage running
Available Repositories 1493
Downloaded Repositories 13
Home Assistant Cloud
logged_in false
can_reach_cert_server ok
can_reach_cloud_auth ok
can_reach_cloud ok
Home Assistant Supervisor
host_os Home Assistant OS 14.1
update_channel stable
supervisor_version supervisor-2024.12.3
agent_version 1.6.0
docker_version 27.2.0
disk_total 28.5 GB
disk_used 20.2 GB
healthy true
supported true
host_connectivity true
supervisor_connectivity true
ntp_synchronized true
virtualization
board rpi3-64
supervisor_api ok
version_api ok
installed_addons Terminal & SSH (9.16.0), File editor (5.8.0), AdGuard Home (5.2.1), eufy-security-ws (1.9.1), ESPHome Device Builder (2024.12.2), Home Assistant Google Drive Backup (0.112.1), Tailscale (0.24.0)
Dashboards
dashboards 4
resources 6
views 7
mode storage
Recorder
oldest_recorder_run December 25, 2024 at 9:39 AM
current_recorder_run January 4, 2025 at 2:47 AM
estimated_db_size 37.20 MiB
database_engine sqlite
database_version 3.45.3

@lmagyar
Copy link
Contributor

lmagyar commented Jan 5, 2025

Hmmm, nothing special, nearly identical with one of my dev boards.

New random idea: rpi3-64 runs out of memory quite soon, what does Settings / System / Hardware / memory graph show? If it is above ~70%, memory can be an issue. Try stopping eg. ESPHome add-on then start TS v0.24.0. TS v0.24.0 uses Alpine 3.21, and maybe that uses a little bit more memory than previous 3.20. I use my live rpi3 systems with 32bit image, that eats up approx. 10-20% les RAM than the 64 bit version. Though not all add-ons run on 32bit. I don't know a proper way to diagnose detect "out of memory" errors on HA OS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants