IrrecoverableError: Error starting child process[alpine-docker issue] #86

charany1 · 2016-12-22T15:10:51Z

Version : amazon-kinesis-producer : 0.12.1

I'm running a Kinesis Producer inside a docker container , and getting this error :

com.amazonaws.services.kinesis.producer.IrrecoverableError: Error starting child process at com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:520) [application.jar:0.1] at com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:453) [application.jar:0.1] at com.amazonaws.services.kinesis.producer.Daemon.access$100(Daemon.java:62) [application.jar:0.1] at com.amazonaws.services.kinesis.producer.Daemon$1.run(Daemon.java:132) [application.jar:0.1] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_92-internal] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_92-internal] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_92-internal] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_92-internal] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_92-internal] Caused by: java.io.IOException: Cannot run program "/tmp/amazon-kinesis-producer-native-binaries/kinesis_producer_f40412a63c9816 d7b4c06e0b1f597c4f3280d36e": error=2, No such file or directory at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048) ~[na:1.8.0_92-internal]

It says that it can't find file in "/tmp/amazon.." , however I'm able to the file available there , what might be the issue ?

The text was updated successfully, but these errors were encountered:

samuelgmartinez · 2016-12-22T15:22:42Z

I had a similar problem running a Kinesis producer daemon within a docker container built from an alpine image.

We fixed it using ubuntu as a base image for that service.

EDIT: grammar (oh god, my english is awful)

charany1 · 2016-12-22T15:25:33Z

@samuelgmartinez aha... same here , thanks a lot , will go with ubuntu .

pfifer · 2016-12-27T15:32:38Z

It appears Alpine Linux is based of musl libc and busybox. The kinesis_producer binary is linked against glibc, so when the loader kicks off it's failing to find correct libc.so and causing this error message. The only way to support Alpine Linux would be to compile the kinesis_producer against musl libc.

BradErz · 2017-05-18T16:37:43Z

@pfifer is there any possibility of getting this fixed? alpine linux is much more docker friendly compared to any ubuntu images.

L3O · 2017-10-27T15:33:00Z

Any updates on this ?

alphafoobar · 2018-04-17T04:31:30Z

I'm seeing something that looks very similar to this issue using amazon-kinesis-producer : 0.12.8 on the openjdk-alpine docker image

com.amazonaws.services.kinesis.producer.IrrecoverableError: Error starting child process
        at com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:525) [amazon-kinesis-producer-0.12.8.jar:na]
        at com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:456) [amazon-kinesis-producer-0.12.8.jar:na]
        at com.amazonaws.services.kinesis.producer.Daemon.access$100(Daemon.java:63) [amazon-kinesis-producer-0.12.8.jar:na]
        at com.amazonaws.services.kinesis.producer.Daemon$1.run(Daemon.java:133) [amazon-kinesis-producer-0.12.8.jar:na]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_151]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_151]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151]
Caused by: java.io.IOException: Cannot run program "/tmp/amazon-kinesis-producer-native-binaries/kinesis_producer_0529847a6647765630c02beeaf6a22c24858f873": error=2, No such file or directory
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048) ~[na:1.8.0_151]
        at com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:454) [amazon-kinesis-producer-0.12.8.jar:na]
        ... 5 common frames omitted
Caused by: java.io.IOException: error=2, No such file or directory
        at java.lang.UNIXProcess.forkAndExec(Native Method) ~[na:1.8.0_151]
        at java.lang.UNIXProcess.<init>(UNIXProcess.java:247) ~[na:1.8.0_151]
        at java.lang.ProcessImpl.start(ProcessImpl.java:134) ~[na:1.8.0_151]
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029) ~[na:1.8.0_151]
        ... 6 common frames omitted

pfifer · 2018-04-17T15:08:33Z

The stack trace you provided is the message you would expect to see for a linked library missing. The The file not found is coming from the inability to load glibc.

jongsy · 2018-05-22T12:26:55Z

I changed the base docker container to frolvlad/alpine-oraclejdk8 which has glibc installed.

anirudh1800 · 2018-07-23T17:45:07Z

Facing the same issue with tomcat:8.0.52-jre8-alpine image.

alphafoobar · 2018-07-23T20:36:28Z

You have a few options...

Continue to use an alpine Linux based openjdk image... And add glibc. We built or own that is available on docker hub https://hub.docker.com/r/bncprojects/openjdk/ and on GitHub https://github.com/bnc-projects/base-openjdk
Use a different openjdk Linux docker image like slim that already has glibc.

simon-katz · 2018-09-21T09:44:47Z

I got around this by switching from alpine to frolvlad/alpine-glibc.

cgpassante · 2019-06-18T20:55:26Z

I tried a bunch of images and selected amazoncorretto:8u212. All exhibited the same behavior: Create a producer, send a record and then DaemonException, recreate the producer and send a record and it works. May not be related to image. Could be some sort of race condition or startup error.

matthewpick · 2020-03-10T18:40:34Z

I ended up using frolvlad/alpine-java:jre8-slim to get around this issue.

sankarnadendla · 2020-04-18T14:34:31Z

I'm seeing something that looks very similar to this issue using amazon-kinesis-producer : 0.12.8 on the openjdk-alpine docker image

com.amazonaws.services.kinesis.producer.IrrecoverableError: Error starting child process
        at com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:525) [amazon-kinesis-producer-0.12.8.jar:na]
        at com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:456) [amazon-kinesis-producer-0.12.8.jar:na]
        at com.amazonaws.services.kinesis.producer.Daemon.access$100(Daemon.java:63) [amazon-kinesis-producer-0.12.8.jar:na]
        at com.amazonaws.services.kinesis.producer.Daemon$1.run(Daemon.java:133) [amazon-kinesis-producer-0.12.8.jar:na]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_151]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_151]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151]
Caused by: java.io.IOException: Cannot run program "/tmp/amazon-kinesis-producer-native-binaries/kinesis_producer_0529847a6647765630c02beeaf6a22c24858f873": error=2, No such file or directory
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048) ~[na:1.8.0_151]
        at com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:454) [amazon-kinesis-producer-0.12.8.jar:na]
        ... 5 common frames omitted
Caused by: java.io.IOException: error=2, No such file or directory
        at java.lang.UNIXProcess.forkAndExec(Native Method) ~[na:1.8.0_151]
        at java.lang.UNIXProcess.<init>(UNIXProcess.java:247) ~[na:1.8.0_151]
        at java.lang.ProcessImpl.start(ProcessImpl.java:134) ~[na:1.8.0_151]
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029) ~[na:1.8.0_151]
        ... 6 common frames omitted

I have got the same exact error, Could you please how to resolve this. Thanks in advance

[kpl-daemon-0000] com.amazonaws.services.kinesis.producer.KinesisProducer - Error in child process com.amazonaws.services.kinesis.producer.IrrecoverableError: Error starting child process at com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:525) [amazon-kinesis-producer-0.12.7.jar!/:?] at com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:456) [amazon-kinesis-producer-0.12.7.jar!/:?] at com.amazonaws.services.kinesis.producer.Daemon.access$100(Daemon.java:63) [amazon-kinesis-producer-0.12.7.jar!/:?] at com.amazonaws.services.kinesis.producer.Daemon$1.run(Daemon.java:133) [amazon-kinesis-producer-0.12.7.jar!/:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_242] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_242] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242] Caused by: java.io.IOException: Cannot run program "/tmp/amazon-kinesis-producer-native-binaries/kinesis_producer_3c24635e636b1b743e5193137a9682a793727d6b": error=2, No such file or directory at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048) ~[?:1.8.0_242] at com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:454) ~[amazon-kinesis-producer-0.12.7.jar!/:?] ... 5 more Caused by: java.io.IOException: error=2, No such file or directory at java.lang.UNIXProcess.forkAndExec(Native Method) ~[?:1.8.0_242] at java.lang.UNIXProcess.<init>(UNIXProcess.java:247) ~[?:1.8.0_242] at java.lang.ProcessImpl.start(ProcessImpl.java:134) ~[?:1.8.0_242] at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029) ~[?:1.8.0_242] at com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:454) ~[amazon-kinesis-producer-0.12.7.jar!/:?] ... 5 more

bcmedeiros · 2021-02-23T03:20:09Z

Same issue here with an image derived from azul/zulu-openjdk-alpine:11.0.9:

2021-02-23 03:05:17.302000 +0000 [kpl-daemon-0000] ERROR com.amazonaws.services.kinesis.producer.KinesisProducer - Error in child process
com.amazonaws.services.kinesis.producer.IrrecoverableError: Error starting child process
        at com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:536)
        at com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:467)
        at com.amazonaws.services.kinesis.producer.Daemon.access$100(Daemon.java:61)
        at com.amazonaws.services.kinesis.producer.Daemon$1.run(Daemon.java:130)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: Cannot run program "/tmp/amazon-kinesis-producer-native-binaries/kinesis_producer_3F8F88A118FB71C891E47599DBE35C90AEB271FF": error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(Unknown Source)
        at java.base/java.lang.ProcessBuilder.start(Unknown Source)
        at com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:465)
        ... 5 more
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(Unknown Source)
        at java.base/java.lang.ProcessImpl.start(Unknown Source)
        ... 8 more

Funny fact is that adoptopenjdk/openjdk11:jre-11.0.9_11.1-alpine works fine, I'm not sure why.

KidCrippler · 2021-04-01T23:35:39Z

Any way to get around this with java 11 and amazoncorretto? It's pretty ironic that of all implementations, the amazon one doesn't gel with kinesis.
I'm building the docker from amazoncorretto:11 and running from amazoncorretto:11-alpine (maybe there's a different jre better suited for this?)

alphafoobar · 2021-04-02T06:45:37Z

Any way to get around this with java 11 and amazoncorretto? It's pretty ironic that of all implementations, the amazon one doesn't gel with kinesis.

I'm building the docker from amazoncorretto:11 and running from amazoncorretto:11-alpine (maybe there's a different jre better suited for this?)

The problem isn't corretto, the problem is alpine. It doesn't have glibc which isn't a dependency for Java, but it is a dependency for the Amazon kinesis native library. You'd be better using an Ubuntu slim docket image.

jaysooo · 2022-03-17T02:23:38Z

in my case..
I used azul/zulu-openjdk-centos base image to solve the problem.
I Think I spent two days with this problem... OTL

Nishant-Pathak · 2024-04-07T15:00:59Z

Is this issue fixed? We have an hard dependency on alpine image

Nishant-Pathak · 2024-04-09T04:18:46Z

We have fixed it by adding below lines in our docker build of the alpine base image:

RUN echo 'https://storage.sev.monster/alpine/edge/testing' | tee -a /etc/apk/repositories \
    && wget https://storage.sev.monster/alpine/edge/testing/x86_64/sevmonster-keys-1-r0.apk \
    && apk add --allow-untrusted ./sevmonster-keys-1-r0.apk \
    && apk update \
    && apk add gcompat \
    && rm /lib/ld-linux-x86-64.so.2 \
    && apk add --force-overwrite glibc \
    && apk add glibc-bin

lhotari · 2024-09-30T11:29:50Z

Mixing real glibc in Alpine will result in an unstable environment.

blog post by Ariadne Conill, there is no such thing as a "glibc based alpine image"
comments explaining reasons:

pratikdandavate · 2024-11-20T07:35:39Z

I was using amazoncorretto:17-alpine earlier, I used to get java out-of-memory error for heap space. I solved it by changing the image to openjdk:17-jdk-slim and it worked for me.

lhotari · 2024-12-18T16:07:03Z

I have created a PR to support building the kinesis_producer binary on Alpine: #620.

charany1 changed the title ~~IrrecoverableError: Error starting child process~~ IrrecoverableError: Error starting child process[alpine-docker issue] Dec 23, 2016

pfifer added the enhancement label Feb 15, 2017

bobzoller mentioned this issue Jul 6, 2017

logstash-output-kinesis does not work inside alpine linux samcday/logstash-output-kinesis#21

Closed

jeff303 mentioned this issue Nov 16, 2017

Explicit dependency on libidn shared library? #158

Closed

fmthoma mentioned this issue Apr 9, 2018

addUserRecord call throws DaemonException #39

Closed

lhotari mentioned this issue Dec 18, 2024

Support building on Alpine Linux musl #620

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IrrecoverableError: Error starting child process[alpine-docker issue] #86

IrrecoverableError: Error starting child process[alpine-docker issue] #86

charany1 commented Dec 22, 2016 •

edited

Loading

samuelgmartinez commented Dec 22, 2016 •

edited

Loading

charany1 commented Dec 22, 2016

pfifer commented Dec 27, 2016

BradErz commented May 18, 2017

L3O commented Oct 27, 2017

alphafoobar commented Apr 17, 2018

pfifer commented Apr 17, 2018

jongsy commented May 22, 2018 •

edited

Loading

anirudh1800 commented Jul 23, 2018

alphafoobar commented Jul 23, 2018

simon-katz commented Sep 21, 2018 •

edited

Loading

cgpassante commented Jun 18, 2019

matthewpick commented Mar 10, 2020

sankarnadendla commented Apr 18, 2020 •

edited

Loading

bcmedeiros commented Feb 23, 2021

KidCrippler commented Apr 1, 2021

alphafoobar commented Apr 2, 2021

jaysooo commented Mar 17, 2022

Nishant-Pathak commented Apr 7, 2024 •

edited

Loading

Nishant-Pathak commented Apr 9, 2024 •

edited

Loading

lhotari commented Sep 30, 2024

pratikdandavate commented Nov 20, 2024

lhotari commented Dec 18, 2024

IrrecoverableError: Error starting child process[alpine-docker issue] #86

IrrecoverableError: Error starting child process[alpine-docker issue] #86

Comments

charany1 commented Dec 22, 2016 • edited Loading

samuelgmartinez commented Dec 22, 2016 • edited Loading

charany1 commented Dec 22, 2016

pfifer commented Dec 27, 2016

BradErz commented May 18, 2017

L3O commented Oct 27, 2017

alphafoobar commented Apr 17, 2018

pfifer commented Apr 17, 2018

jongsy commented May 22, 2018 • edited Loading

anirudh1800 commented Jul 23, 2018

alphafoobar commented Jul 23, 2018

simon-katz commented Sep 21, 2018 • edited Loading

cgpassante commented Jun 18, 2019

matthewpick commented Mar 10, 2020

sankarnadendla commented Apr 18, 2020 • edited Loading

bcmedeiros commented Feb 23, 2021

KidCrippler commented Apr 1, 2021

alphafoobar commented Apr 2, 2021

jaysooo commented Mar 17, 2022

Nishant-Pathak commented Apr 7, 2024 • edited Loading

Nishant-Pathak commented Apr 9, 2024 • edited Loading

lhotari commented Sep 30, 2024

pratikdandavate commented Nov 20, 2024

lhotari commented Dec 18, 2024

charany1 commented Dec 22, 2016 •

edited

Loading

samuelgmartinez commented Dec 22, 2016 •

edited

Loading

jongsy commented May 22, 2018 •

edited

Loading

simon-katz commented Sep 21, 2018 •

edited

Loading

sankarnadendla commented Apr 18, 2020 •

edited

Loading

Nishant-Pathak commented Apr 7, 2024 •

edited

Loading

Nishant-Pathak commented Apr 9, 2024 •

edited

Loading