[BUG] latest image (10.9.9) stuck in a boot loop #261
Comments
|
Thanks for opening your first issue here! Be sure to follow the relevant issue templates, or risk having this issue marked as invalid. |
|
There is no errors on the pasted log. logs from a fresh run(with our ascii logo) might help |
|
The error message is "Fatal glibc error: cannot get entropy for arc4random" Seems like an issue with the (old) kernel version a Synology NAS is using.
|
|
I have the same behavior. The error message is only visible in the "Terminal" but not in the Jellyfin log files itself. Synology is running on an older Kernel 3.10 even with the latest DSM. |
|
Little we can do if this is kernel related. |
|
Synology don't update their kernels after launch, so you're stuck with 3.10, which at this point is about 7 years past EOL, and will increasingly struggle with anything expecting to be able to leverage newer kernel functionality. |
|
The problem doesn´t seem to occur when using the version with container tag "version-10.9.9ubu2204" . This is the version with an ubuntu 22.04 as base image. So the combination of the Synology kernel icm with new ubuntu 24.04 base image that is used for jellyfin docker container seems to be the issue. |
|
Experiencing the same behavior on a Synology DS916+ 8GB. On my newer Synology DS923+ is runs fine. |
|
I have the same issue on my Synology DS916+ 8GB with the latest version. |
|
Rolling back to an older version is only a workaround, not a fix. We aren't going to roll back our rebase of the container as it's just kicking the can down the road. |
|
Can you explain to me what the gain for Jellyfin is when using ubuntu 24.04 instead of ubuntu 22.04 as a base image ? By using the ubuntu 24.04 base a lot of Synology users will be excluded. |
|
The TL;DR is new OS versions mean updated packages mean better performance and new functionality. Every additional base image we have to maintain is a bunch of admin overhead. So, when a new version has been out for a while and is stable, we look to move everything we can to use it. In theory we can keep using Jammy until 2027 but in practice we want to avoid that. Digging into this issue in detail it looks like the arc4random function was added in glibc 2.36 and might not be inherently tied to the kernel itself. That said, Synology are appalling at updating their core platforms after release and essentially abandon them as soon as they're out the door, so I can't imagine this ever being fixed. We may reconsider our position depending on the number of affected users and whether it extends beyond just the Jellyfin image, but this is not the first time 3.x kernel Synology boxes have run into problems with images and won't be the last. Again, it's been EOL for 7 years now and the 3.10 branch was originally released 11 years ago. |
|
It does appear that @erik-de-bont has a point. Digging through the tags on Docker Hub, it appears that only starting with this specific version of Jellyfin 10.9.9 did you start working with an Ubuntu 24.04 based image. Even all the images for Jellyfin 10.9.8 only show Ubuntu 22.04. This means that only with 10.9.9 did you switch the "latest" tag to point to a 24.04 based image. And you still thankfully DO have Ubuntu 22.04 based images (based on the tags), indicating at least that you realize this may present an issue somewhere. And while I totally get "The TL;DR is new OS versions mean updated packages mean better performance and new functionality" bit, the counterpoint to that is "What exactly are you getting in Ubuntu 24.04 relative to running a Jellyfin container that benefits you over Ubuntu 22.04? Or are you simply chasing 'teh shiny' here?" There is something to be said for "If it ain't broke, don't fix it." One could argue Jellyfin is written in C#. What did Ubuntu 24.04 LTS bring to the table over 22.04 that benefits THAT? I myself prefer and run Ubuntu LTS releases on my own systems. But even I don't move that quickly to using the latest/greatest LTS releases. Ubuntu LTS releases are good for 5 years. And I typically wait 6 months to a year before upgrading any of my older LTS releases, first letting others kink out the inevitable glitches. And that is on an underlying OS. For container images, I typically gun for the simplest base image I can that will work to guarantee maximum compatibility. But hey, that's me. In the end, this is your project. And to be clear, first and foremost I am grateful that you offer this up in any form. It made it possible for me to even try Jellyfin in my setup. So if you all have decided you NEED 24.04 as your base, then I may well just be S.O.L. I am trying to run Jellyfin specifically ON my Synology because that's where my media is stored. If I have to start running Jellyfin on another server/VM, where it then has to use the network to connect to a file share to pull the media content across the network, simply to turn around and send it back out across the same network to whatever client someone is on that is watching, that is a suboptimal config. I'm hairpinning everything, and in this case being video it's a lot of churn for no real benefit. But maybe the Venn diagram of Synology owners and Jellyfin users isn't that big. I honestly don't know. I just know that until this update, it ran incredibly well, and I very much appreciated having it. Anyway, for the moment, I have used @erik-de-bont 's info to manually download the image with tag "version-10.9.9ubu2204". I then cloned my config, tweaked the tag setting to use this image, and after dealing with one more issue (you can't have 2 containers mapped to the same host posts), sure enough, it's working. Not ideal from an updating perspective, but it works. To date I simply used If nothing else, I would like to recommend a feature request: that you possibly add another tag for this. That is, currently |
|
I understand the frustration, but it should really be directed to Synology here. The fact that they keep their NASes on kernel 3.X in 2024 is abhorrent. We will not let Syno's archaic kernels hold our images back. All of our images get rebased on the latest stable or LTS baseimages once we deem the OS stable in our internal testing and confirm the upstream app works. That's been our operating procedure for years. We are actively going through the list for the ubuntu based images as we speak. Once they are all moved over to noble, we will deprecate jammy like we deprecated focal before that. |
|
If you want the numbers we have 29 Ubuntu-based images (not including Kasm derivatives which use a different base) of which 9 remain on Jammy, and those are a mix of images we haven't got around to yet and images that have some dependency issue preventing the move to Noble. As I said it's not about moving Jellyfin specifically, it's about moving all of our images so that we aren't having to maintain lots of bases; if it hadn't been kicked down the road by a couple of weeks, 24.04.1 would have been released this week, which is their "upgrade all your existing stuff" build - not that we're technically upgrading anything. Having a separate image tag for Jellyfin using the Jammy base would be more overhead and not worth the extra work; if it gets to that point we would prefer to roll back the main image to Jammy. We will continue to keep an eye on this issue and if it becomes necessary to change our position on it we will. Because I'm an idiot I went and pulled all our stats for the Jellyfin image; of the 131,727 unique users in the last 3 months, 150 of them were using devices with a 3.10 kernel. Can't say for sure how many are Synology, but my best guess is about 2/3 of them. |
Sadly, no argument. Well there are, but no point going into them. But this is a well worn path with little change expected. Synology likely will argue that they focus on stability over "latest hotness", and as their use case is specific, can't really argue too much. It's a highly customized kernel, where they've stripped out all the unneeded bits. And they backport whatever vulnerability changes they need to. So not a "stock kernel" by any measure. Can't really compare that with a basic distro install. So if the reasons for calling it "abhorrent" are along the lines of "it's old so it must be unsafe!", not a solid argument. That it doesn't have the latest/greatest features, sure. But again, what exactly does a media system get out of the latest LTS release that it didn't out of the previous one? But this is one of those "round and round we go" sort of discussions. So will leave it alone. Heck, Synology could just as easily NOT offer any kind of Docker support. So at least they have something there which let me do this for as long as I have so far. But sounds like I may well have to consider alternatives. (Truth is I've been keeping my eyes on alternative NAS solutions (e.g., UGREEN's recent foray) for my next setup, as the argument that Synology gear is overpriced for the hardware you get is very valid. But to date the Synology has worked well, both in hardware and the software they offer. So hard to argue with it. This is, after, just a hobby setup.) And @thespad , totally get all that. Only so many hours in a day. And in the end, it's a lot of volunteer work. So I'm grateful for what all you folks do regardless. ANY FLOSS project has my respect, as it means something where before there was nothing. So thanks for everything you all do. |
|
The real problem is that they keep launching new hardware with what is essentially already an EOL kernel. The DS416Play was released in mid 2016 with the 3.10 kernel, which went EOL in 2017, for example. It's all very well backporting security fixes, but there can be significant functional changes between major kernel versions and docker is unfortunately one of the main ways that's exposed. Interestingly it looks like DSM 7.2 was supposed to bump glibc to 2.36, which suggests it is a kernel limitation as well as a glibc one, because 2.36 should support the arc4random function. Not sure if you can confirm the version. Like I said, in theory we can keep using Jammy until early 2027, but in 2.5 years we're still going to have people with Synology kit on 3.10. We know this because we've still got users running kit with CPUs from 2008, somehow, as they ran into issues recently due to changes in a certbot dependency that was picked up by our swag image. But again, just to clarify, it's not about Jellyfin specifically, it's about all our Ubuntu-based images. While Jellyfin might not get any direct benefit from moving to Noble, other images do. The Ubuntu LTS builds are, by design, very slow to update versions of tools and libraries, so a few years in you're way behind the curve. Jammy shipped with nodejs 12, which went EOL 3 years ago, Python 3.10, which is end of active support and will go fully EOL before Jammy does, and Ruby 3.0, which went EOL almost 6 months ago. |
|
My Synology DS216+II still has a few years left to run, so I decided to switch to the official Jellyfin container, which has a Debian 12 base image. A few modifications to the path structure of Jellyfin config and it is up and running again. I am of the opinion that a base image should serve the application and nothing more. Therefore, I consider a Debian 12 stable base image more suitable for this purpose. I understand the wish to use only one base image for the containers. I only think Ubuntu is the wrong choice. It is just a different mindset (stability over features) Thank you, however, for all the effort you have put in the project and until we'll meet again! |
|
I'm on a DS916+ which is not about to retire only because linuxserver's special base image strategy (see @erik-de-bont above). I still highly appreciate your work, but now it's time for me to move on to some more continuous / less-progessive image provider. I expect this also to be the case for all my other linuxserver-images, which is a pity... |
|
I am on UNRAID and this is happening to me as well. Soon as it updated its like it boots over and over raising usage memory etc.. then it hangs and acts very wonky on the tv. Tried rolling back but it still does it. Maybe ill try older image than 10.9.9 , however my web and build version both still show 10.9.10 |
|
Hey guys, I'm not concerned by this issue, as my (slightly newer) Synology is running kernel 4.4. Just wanted to know if anyone has an idea of how "future-proof" this is in regards to ubuntu image versions for linuxserver builds? |
|
Short answer is you can't future-proof a system that will never get updated; Synology kernels are locked in stasis from the point of release, getting only backported security and critical bug fixes. Sooner or later stuff will stop working. You can version pin your container images, but that has its own drawbacks because you stop getting application and OS updates. As I mentioned before, in terms of the 3.10 kernel we've got about 0.1% of Jellyfin users on it, so it's hard to justify making changes just for that. For the 4.4 kernel it's about 1.5% of Jellyfin users so the maths aren't really great there either. We obviously don't want to break older systems but we have to be pragmatic when we're managing as many images as we do (and sometimes it's unavoidable in any case). |
|
@hundredfire , as @thespad indicates, there's really no true "future proofing", as things are constantly changing/updating. DSM will update at its pace. Linuxserver updates at theirs. When things get too far apart, we get situations like this. But until this happened, the Linuxserver images served me very well. In the end, like @hoewer, I opted for replacing the Linuxserver image with the hotio one, which is working without issue. And as I can see this same issue may start affecting other containers I'm running on the same Synology DS916+, as I don't see Synology updating the DSM kernel version on that per se, I right away moved my other Linuxserver image based apps to their official images as well. So at this point my Synology is no longer running any Linuxserver images. To be clear, that's not disparaging of the Linuxserver folks. They have their reasons and they're doing an awful lot of good work for folks. I'm just sitting on some older gear and it's clearly too small a % to matter to how the Linuxserver folks do things. So I did what I had to do to maintain stability in my setup. And yes, the images I've chosen don't impose the same problem because they don't require the extra bits that apparently the Ubuntu 24.04 LTS base OS image does. Does this mean I have "future proofed" my setup? No, not really. Right now I have a solution which doesn't impose the same particular limitation that the Linuxserver base OS cadence does. But that doesn't mean I won't get hit with something else at some point, as the challenge here is that the underlying OS (DSM) is the one with the oldest bits in it. All we can really do is try our best to find solutions that hopefully last long enough in our situations. As you indicate, newer Synologies apparently have newer kernel versions. That's great. (I'm curious which model you have and what version of DSM you're running.) So this whole thread may be a non-issue to you for some time still. Anyway, that ended up being my solution. I just wanted to pop back here and thank the Linuxserver folks for all the hard work they've done/do. It's still very much appreciated, even if I can't use it here for the time being. |
|
Thanks for the replies @thespad and @fseesink (for info, I'm running a Synology DS220+, so not a very recent model neither). |
|
This issue has been automatically marked as stale because it has not had recent activity. This might be due to missing feedback from OP. It will be closed if no further activity occurs. Thank you for your contributions. |
Is there an existing issue for this?
Current Behavior
I just updated the
linuxserver/jellyfin:latest(which appears to be 10.9.9 based on the Docker Hub page showing the timestamps for images) and now I can no longer run Jellyfin. Checking the logs, it appears to be stuck in a boot loop. Logs show this pattern repeating over and over again:Expected Behavior
I expected to update Jellyfin and run this version.
Steps To Reproduce
Pulled
linuxserver/jellyfin:latestdown, stopped existing container, replace it, and started it back up.Environment
CPU architecture
x86-64
Docker creation
Container logs
The text was updated successfully, but these errors were encountered: