npm:docker-modem doesn't work (and @testcontainers/* as a result)

[gflarity]

The blog post said we should report, so here we go:

I just trying to use testcontainers (@testcontainers/postgresql) with Deno when I got this strange error:

error: Error: (HTTP code 400) unexpected - starting container with non-empty request body was deprecated since API v1.22 and removed in v1.24
    at file:///Users/geoff/Library/Caches/deno/npm/registry.npmjs.org/docker-modem/3.0.8/lib/modem.js:343:17
    at getCause (file:///Users/geoff/Library/Caches/deno/npm/registry.npmjs.org/docker-modem/3.0.8/lib/modem.js:373:7)
    at Modem.buildPayload (file:///Users/geoff/Library/Caches/deno/npm/registry.npmjs.org/docker-modem/3.0.8/lib/modem.js:342:5)
    at IncomingMessageForClient.<anonymous> (file:///Users/geoff/Library/Caches/deno/npm/registry.npmjs.org/docker-modem/3.0.8/lib/modem.js:310:16)

So it seems there might be some http client compatibility issues, as the docker api is just HTTP last time I checked...

Here's the repro code which was inspired by this Testcontainer Node quickstart repo (which runs fine using node):

import { PostgreSqlContainer } from "npm:@testcontainers/postgresql";
import postgres from "https://deno.land/x/postgresjs/mod.js";

Deno.test("hello postgres", async () => {
  const initScript = `
            create table guides
            (
                id    bigserial     not null,
                title varchar(1023) not null,
                url   varchar(1023) not null,
                primary key (id)
            );

            insert into guides(title, url)
            values ('Getting started with Testcontainers',
                    'https://testcontainers.com/getting-started/'),
                   ('Getting started with Testcontainers for Java',
                    'https://testcontainers.com/guides/getting-started-with-testcontainers-for-java/'),
                   ('Getting started with Testcontainers for .NET',
                    'https://testcontainers.com/guides/getting-started-with-testcontainers-for-dotnet/'),
                   ('Getting started with Testcontainers for Node.js',
                    'https://testcontainers.com/guides/getting-started-with-testcontainers-for-nodejs/'),
                   ('Getting started with Testcontainers for Go',
                    'https://testcontainers.com/guides/getting-started-with-testcontainers-for-go/'),
                   ('Testcontainers container lifecycle management using JUnit 5',
                    'https://testcontainers.com/guides/testcontainers-container-lifecycle/')
            ;
        `;

  const container = await new PostgreSqlContainer("postgres:14-alpine",)
    .withCopyContentToContainer([
      { content: initScript, target: "/docker-entrypoint-initdb.d/init.sql" },
    ])
    .start();

  const sql = postgres(container.getConnectionUri());
  const res = await sql`select title, url from guides`;
  console.log(res);
});

[brad-jones]

I just ran into something similar. When I debugged I saw requests being made to http://localhost instead of /var/run/docker.sock.
Looks very similar to apocas/dockerode#747

[dionjwa]

I am also hitting this. Is there any workaround?

[dionjwa]

I adapted the workaround proposed by apocas/dockerode#747 (comment)
to use pure deno and avoid the socat dependency, it works in my own tools:

/******************************************************
 * Begin workarounds for this showstopper issue:
 * https://github.com/apocas/dockerode/issues/747
*/
async function startProxy() {
  // Listen on TCP port 3000
  const tcpListener = Deno.listen({ port: 3000 });
  console.log("Listening on TCP port 3000");

  for await (const tcpConn of tcpListener) {
    handleConnection(tcpConn);
  }
}
async function handleConnection(tcpConn: Deno.Conn) {
  try {
    // Connect to the Unix socket at /var/run/docker.sock
    const unixConn = await Deno.connect({ transport: "unix", path: "/var/run/docker.sock" });

    // Bidirectional forwarding
    const tcpToUnix = copy(tcpConn, unixConn);
    const unixToTcp = copy(unixConn, tcpConn);

    // Wait for both copy operations to complete
    await Promise.all([tcpToUnix, unixToTcp]);

  } catch (error) {
    console.error("Error handling connection:", error);
  } finally {
    tcpConn.close();
  }
}
// Utility function to copy data from one stream to another
async function copy(src: Deno.Reader, dst: Deno.Writer) {
  const buffer = new Uint8Array(1024);
  while (true) {
    const bytesRead = await src.read(buffer);
    if (bytesRead === null) break;
    let offset = 0;
    while (offset < bytesRead) {
      const bytesWritten = await dst.write(buffer.subarray(offset, bytesRead));
      offset += bytesWritten;
    }
  }
}
// Start the proxy
startProxy();
// and this now needs to be changed because of the above:
// const docker = new Docker({socketPath: "/var/run/docker.sock"});
const docker = new Docker({protocol: 'http', host: 'localhost', port: 3000});
/******************************************************
 * End workarounds for this showstopper issue:
 * https://github.com/apocas/dockerode/issues/747
*/

[felipecrs]

This is the only thing keeping me from adopting Deno in one of my projects btw (lack of testcontainers support).

[Karrq]

Thanks @dionjwa, was hitting the same issue on Deno 2, but thanks to the workaround I can continue!

[felipecrs]

I've tried to combine @dionjwa's solution to work with testcontainers:

// docker-proxy.ts

import type { Reader, Writer } from "jsr:@std/io@0.225";

export async function startProxy() {
  const tcpListener = Deno.listen({ port: 2375 });

  for await (const tcpConn of tcpListener) {
    handleConnection(tcpConn);
  }
  return tcpListener;
}

async function handleConnection(tcpConn: Deno.Conn) {
  try {
    // Connect to the Unix socket at /var/run/docker.sock
    const unixConn = await Deno.connect({
      transport: "unix",
      path: "/var/run/docker.sock",
    });

    // Bidirectional forwarding
    const tcpToUnix = copy(tcpConn, unixConn);
    const unixToTcp = copy(unixConn, tcpConn);

    // Wait for both copy operations to complete
    await Promise.all([tcpToUnix, unixToTcp]);
  } catch (error) {
    console.error("Error handling connection:", error);
  } finally {
    tcpConn.close();
  }
}

// Utility function to copy data from one stream to another
async function copy(src: Reader, dst: Writer) {
  const buffer = new Uint8Array(1024);
  while (true) {
    const bytesRead = await src.read(buffer);
    if (bytesRead === null) break;
    let offset = 0;
    while (offset < bytesRead) {
      const bytesWritten = await dst.write(buffer.subarray(offset, bytesRead));
      offset += bytesWritten;
    }
  }
}

// main.ts

import { GenericContainer } from "testcontainers";

import { startProxy } from "./docker-proxy.ts";

startProxy();

Deno.env.set("DOCKER_HOST", "tcp://localhost:2375");

const container = new GenericContainer("alpine");
const startedContainer = await container.start();
await startedContainer.stop();

Then:

❯ deno run -A main.ts
error: Uncaught (in promise) TypeError: stream.socket.unref is not a function
    at DockerContainerClient.logs (file:///home/felipecrs/.cache/deno/npm/registry.npmjs.org/testcontainers/10.13.2/build/container-runtime/clients/container/docker-container-client.js:165:27)
    at eventLoopTick (ext:core/01_core.js:175:7)
    at async LogWaitStrategy.waitUntilReady (file:///home/felipecrs/.cache/deno/npm/registry.npmjs.org/testcontainers/10.13.2/build/wait-strategies/log-wait-strategy.js:22:24)
    at async waitForContainer (file:///home/felipecrs/.cache/deno/npm/registry.npmjs.org/testcontainers/10.13.2/build/wait-strategies/wait-for-container.js:8:9)
    at async GenericContainer.startContainer (file:///home/felipecrs/.cache/deno/npm/registry.npmjs.org/testcontainers/10.13.2/build/generic-container/generic-container.js:143:9)
    at async createNewReaper (file:///home/felipecrs/.cache/deno/npm/registry.npmjs.org/testcontainers/10.13.2/build/reaper/reaper.js:62:30)
    at async file:///home/felipecrs/.cache/deno/npm/registry.npmjs.org/testcontainers/10.13.2/build/reaper/reaper.js:29:20
    at async withFileLock (file:///home/felipecrs/.cache/deno/npm/registry.npmjs.org/testcontainers/10.13.2/build/common/file-lock.js:41:16)
    at async getReaper (file:///home/felipecrs/.cache/deno/npm/registry.npmjs.org/testcontainers/10.13.2/build/reaper/reaper.js:19:14)
    at async GenericContainer.start (file:///home/felipecrs/.cache/deno/npm/registry.npmjs.org/testcontainers/10.13.2/build/generic-container/generic-container.js:75:28)

Which refs to a function not yet implemented apparently:

• https://github.com/testcontainers/testcontainers-node/blob/08da47baeaa9a43f29aec3a9bb1ce67a3bc1849f/packages/testcontainers/src/container-runtime/clients/container/docker-container-client.ts#L168

I believe it could be related to this issue:

• node: implement dgram.Socket APIs #18324

Which is a different compatibility issue.

[mortezaPRK]

I managed to get it working by mocking Socket.unref and reusing @dionjwa proxy:

import { ClientRequest } from "node:http";
import * as io from "jsr:@std/io";

const originalSocket = ClientRequest.prototype.onSocket;
ClientRequest.prototype.onSocket = function (socket) {
    socket.unref = function () { return this; };
    return originalSocket.call(this, socket);
}
async function startProxy() {
    const tcpListener = Deno.listen({ port: 3000 });
    for await (const tcpConn of tcpListener) {
        handleConnection(tcpConn);
    }
}
async function handleConnection(tcpConn: Deno.Conn) {
    try {
        const unixConn = await Deno.connect({ transport: "unix", path: "/var/run/docker.sock" });
        await Promise.all([
            io.copy(tcpConn, unixConn),
            io.copy(unixConn, tcpConn),
        ]);
    } catch (error) {
        console.error("Error handling connection:", error);
    } finally {
        tcpConn.close();
    }
}