feat: Add Google Cloud Storage API (fake-gcs-server) module

[KSemenenko]

What does this PR do?

Add Google Cloud Storage to testcontainer

Why is it important?

Now testcontainer will have Azure, AWS and GCP for tests

Related issues

• Closes [Enhancement]: Google Cloud Storage API #1022

[netlify]

✅ Deploy Preview for testcontainers-dotnet ready!

Name	Link
🔨 Latest commit	5bdc1d7
🔍 Latest deploy log	https://app.netlify.com/sites/testcontainers-dotnet/deploys/653027a80504640008e0569b
😎 Deploy Preview	https://deploy-preview-1023--testcontainers-dotnet.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[KSemenenko]

there is one problem

    return base.Init()
            .WithImage(GCSImage)
            .WithPortBinding(GCSPort, newPort)
            .WithCommand("-scheme", "http")
            .WithCommand("-backend", "memory")
            .WithCommand("-external-url", $"http://localhost:{newPort}")
            .WithCommand("-port", newPort.ToString())
            .WithWaitStrategy(Wait.ForUnixContainer().UntilHttpRequestIsSucceeded(request =>
                request.ForPath("/").ForPort(newPort).ForStatusCode(HttpStatusCode.NotFound)));

I have to pass acutal port into -external-url paramenter.
how can I do this?

[KSemenenko]

this is how docker config shoud looks like
https://github.com/fsouza/fake-gcs-server/blob/main/examples/dotnet/docker-compose.yaml

[HofmeisterAn]

I have to pass acutal port into -external-url paramenter.

Hi 👋 do you mean the actual random assigned host port? It is not very common that a services running inside a container requires it, but you can configure something similar like Kafka and utilize the WithStartupCallback builder member:

testcontainers-dotnet/src/Testcontainers.Kafka/KafkaBuilder.cs

Lines 65 to 66 in 44a1d3a

	.WithEntrypoint("/bin/sh", "-c")
	.WithCommand("while [ ! -f " + StartupScriptFilePath + " ]; do sleep 0.1; done; " + StartupScriptFilePath)

The WithStartupCallback builder member prepares a shell script and copies it during the start to the container. The callback has access to the random assigned host port (see L:82):

testcontainers-dotnet/src/Testcontainers.Kafka/KafkaBuilder.cs

Lines 82 to 87 in 44a1d3a

	startupScript.Append("export KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://" + container.Hostname + ":" + container.GetMappedPublicPort(KafkaPort) + ",BROKER://" + container.IpAddress + ":" + BrokerPort);
	startupScript.Append(lf);
	startupScript.Append("echo '' > /etc/confluent/docker/ensure");
	startupScript.Append(lf);
	startupScript.Append("/etc/confluent/docker/run");
	return container.CopyAsync(Encoding.Default.GetBytes(startupScript.ToString()), StartupScriptFilePath, Unix.FileMode755, ct);

[KSemenenko]

Yeah, it’s because of container, I have no idea why.
But super thanks a lot I will try!

[KSemenenko]

@HofmeisterAn the problem is I have to pass port number into paramter

so for code .WithPortBinding(GCSPort, true)
I have to get somehow this random port and pass it as RandomPort
.WithCommand("-external-url", $"http://localhost:{RandomPort}")
in parameter.
so this is the issue

[KSemenenko]

if I do like hits

return base.Init()
            .WithImage(GCSImage)
            .WithPortBinding(GCSPort, GCSPort)
            .WithCommand("-scheme", "http")
            .WithCommand("-backend", "memory")
            .WithCommand("-external-url", $"http://localhost:{GCSPort}")
            .WithCommand("-port", $"{GCSPort}")
            .WithWaitStrategy(Wait.ForUnixContainer().UntilHttpRequestIsSucceeded(request =>
                request.ForPath("/").ForPort(GCSPort).ForStatusCode(HttpStatusCode.NotFound)));

then all is fine, and it works. but it's not possible to run several containers at the same time

[HofmeisterAn]

@HofmeisterAn the problem is I have to pass port number into paramter

As mentioned in my previous comment, you need a configuration similar to Kafka:

1. Override the entrypoint that executes a shell.
2. Override the command and set a command that waits until the startup script is written and then executes it.
3. Prepare the startup script in WithStartupCallback, where you have access to the random assigned host port.
4. Finally, copy the prepared startup script to the container. As soon as it is written, the overridden entrypoint and respective command will execute it, starting the service inside the container.

[KSemenenko]

ok if I do [Collection("Google")] for collections thet all is fine

[HofmeisterAn]

ok if I do [Collection("Google")] for collections thet all is fine

I am sorry, I do not understand. Modules must work with random assigned host ports; we cannot accept the pull request using fixed host ports. You can find further details in our best practices.

[KSemenenko]

@HofmeisterAn that was my random thoughts, anyway thanks for your help, now it works fine =)
also I maked right tests.

[KSemenenko]

@HofmeisterAn the problem is I have to pass port number into paramter

As mentioned in my previous comment, you need a configuration similar to Kafka:

1. Override the entrypoint that executes a shell.
2. Override the command and set a command that waits until the startup script is written and then executes it.
3. Prepare the startup script in WithStartupCallback, where you have access to the random assigned host port.
4. Finally, copy the prepared startup script to the container. As soon as it is written, the overridden entrypoint and respective command will execute it, starting the service inside the container.

This is really amazing suggeest! thanks a lot!

[KSemenenko]

Now it’s ready for review

[HofmeisterAn]

Thanks for the PR 🙌. I have only a few minor suggestions to improve the module. In addition to my comments below, can we rename GCS to Gcs to align with the other modules?

[KSemenenko]

fixed, but with some reason one pipline is borken, I did that?

[KSemenenko]

aH I see

[KSemenenko]

@HofmeisterAn thansks a lot! This is super cool finding!

[KSemenenko]

@HofmeisterAn Hi! Thanks for merging this request!
By the way, when will his package be available?

[HofmeisterAn]

I have not planned a release yet. I want to make some minor changes in the next days, but I need a short break as I am feeling drained.

[KSemenenko]

Sure! I'm just checking to see when I can throw out my old implementation and replace it with a nice nuget package =)