Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Race condition in docker.start #582

Open
ckanibal opened this issue Jul 21, 2020 · 1 comment
Open

Race condition in docker.start #582

ckanibal opened this issue Jul 21, 2020 · 1 comment

Comments

@ckanibal
Copy link

First of all, Thank You! for this amazing project.

When I try to run a short running operation in a new container with the HostOption AutoRemove: true like this:

docker.run(options.image,
        [command, ...args],
        [stdout, stderr],
        {
            Tty: false,
            HostConfig: {
                AutoRemove: true,
                ...(options.customCwd && [`${options.customCwd}:${options.customCwd}:rw`])
            },
        })
        .then(([res, container]) => {
            logger.debug("Docker run successful", {res, container});
            return {
                code: res.StatusCode,
                okToCache,
                filenameTransform,
                stdout: stdout.toString(),
                stderr: stderr.toString(),
            };
        })
        .catch(error => {
            logger.warn("Docker run failed", {error});
            return error;
        });

it occasionally fails with the error message:

Error: (HTTP code 404) no such container - No such container: 919455c4b42dd72b6767c129772a134616d6d8d8e7b54e9f947a0d92b74f2fa2 
    at /project/node_modules/docker-modem/lib/modem.js:301:17
    at getCause (/project/node_modules/docker-modem/lib/modem.js:331:7)
    at Modem.buildPayload (/project/node_modules/docker-modem/lib/modem.js:300:5)
    at IncomingMessage.<anonymous> (/project/node_modules/docker-modem/lib/modem.js:275:14)
    at IncomingMessage.emit (events.js:333:22)
    at IncomingMessage.EventEmitter.emit (domain.js:485:12)
    at endReadableNT (_stream_readable.js:1220:12)
    at processTicksAndRejections (internal/process/task_queues.js:84:21)

I tracked the issue down to those lines of code:

dockerode/lib/docker.js

Lines 1486 to 1494 in ed6ef39

container.start(startOptions, function(err, data) {
if (err) return callback(err, data, container);
hub.emit('start', container);
container.wait(function(err, data) {
hub.emit('data', data);
callback(err, data, container);
});
});

The wait operation can fail, if the container is already finished and automatically removed between the start and wait operation. I have no good idea how to mitigate this issue, yet. Somebody care to take a look?

@apocas
Copy link
Owner

apocas commented Apr 5, 2021

Interesting. But I don't see an easy fix for this one.

Checking the container's status will not help. We can't assume the container finished correctly after a very fast lifecycle, something could went bad.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants