-
Notifications
You must be signed in to change notification settings - Fork 134
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Graylog fails to run after updating docker image from 4.3.3 to 4.3.4 #217
Comments
@MahdiGhiasi |
This seems to be where this happens: https://github.com/Graylog2/graylog-docker/blob/4.3/docker-entrypoint.sh#L91-L94
|
We're currently running 2 environments on the
|
@mpfz0r Here's the output for the debugging entry point you requested:
|
I've also confirmed again that docker image I've rolled back our production Graylog instance to 4.3.3 for now and it runs properly; but I'm happy to do any tests or give any details that might help finding the root cause of this in 4.3.4 on our environment. |
@MahdiGhiasi Thanks. Could you run |
@mpfz0r There you go:
It seems that @coffee-squirrel is right, the last few lines seem to indicate that |
@MahdiGhiasi Yeah, looks like it. But your output is truncated. It misses the part where /etc/profile is sourced, that's where GRAYLOG_HOME should be set |
@mpfz0r Oh, sorry. Here's the complete output:
|
@MahdiGhiasi what is the output of |
|
@MahdiGhiasi |
|
@MahdiGhiasi |
@mpfz0r This does not print |
@MahdiGhiasi hmm, that's weird.. And for now I'm out of ideas. Maybe some one else? |
@mpfz0r Docker version 20.10.5 (build 55c4c88), running on Ubuntu 20.04.1 LTS. On a maybe related note, I've also tried to upgrade another machine to Graylog 4.3.4 from 4.3.3, this one also fails but for an entirely different reason! (This machine is running Docker version 20.10.8 build 3967b7d, Ubuntu 20.04.2 LTS) This one passes the However, heap size is set to 1.5GB ( And this one also works fine on 4.3.3, but breaks on 4.3.4. Here's the log for that:
|
Since you're on Docker 20.10.5 (released 2021-03-02) and 20.10.8 (released 2021-08-03), I'd suggest trying to get that upgraded to at least 20.10.10 (released 2021-10-25; latest is 20.10.17 released 2022-06-06). Doing a bit of searching on the second issue, I found https://stackoverflow.com/a/72841934 (and therefore adoptium/containers#215), which seems like it could be related (potentially to both issues) given |
What @coffee-squirrel said, except that we have also noticed this behavior with alpine based temurin images. |
@MahdiGhiasi |
@mpfz0r I think the profile-script thing is a red herring. The root cause is the fact that the new eclipse-temurin base-image uses a new glibc version that uses the clone3 syscall, which is blocked by docker's default seccomp policy, which was updated with 20.10.10. so all older docker versions will fail with the same issue. If you are able to downgrade your docker version to 20.10.9 or older you should be able to reproduce this issue. |
@pschichtel Thanks! That makes a lot of sense. For reference, it's this comment: adoptium/containers#215 (comment) in particular OK, so I guess we can close this case. Unless we need to support docker versions older than |
Can you update your docker version? |
I can confirm that updating docker to version 20.10.17 solves both issues we were facing. |
Great. I'm gonna keep this open in case more people are running into this. |
We've updated our Graylog docker image to latest version pushed on
graylog/graylog:4.3
(which is 4.3.4), and since then the graylog container fails to start.Inspecting docker logs shows that this seems to be the issue:
Our elastic and mongo are running fine, but graylog container fails to start with the error message above.
We have two separate instances of Graylog (not linked, used separately by separate teams), and both have encountered this issue after upgrading. They were both working fine on v4.3.3.
We use a folder-mounted journal folder in our docker-compose file:
Update (by @mpfz0r):
This only happens on docker versions older than
20.10.10
https://docs.docker.com/engine/release-notes/#201010Because their default seccomp policy does not support
clone3
syscalls.The text was updated successfully, but these errors were encountered: