Start up individual containers/services instead of the whole cluster.

[ziminer]

We are trying to use docker-compose-rule to start up individual containers/services (and, of course, their dependencies, which docker does automatically), and I can't tell if there is already a way to do this. Ideally it would be something like:

DockerComposeRule.builder()
.files(...)
.services("my-service", "other-service")
...
.build()

I saw there is a ".containers" option in the builder, but it takes in a Cluster, and creating a cluster implementation seems overly heavyweight, so I'm not sure if that's it's intended use case.

[joelea]

Hey @ziminer,

This is not something that is supported in the way you're looking for. This is partially because it's not been necessary before, and partially because it's actually kind of fiddly. e.g. docker-compose down does not take container arguments, so you could get into a case where you spin up foo and bar separately, but then the rule takes both of them down afterwards.

The way my current project handles this is by splitting the docker-compose.yml files into individual clusters of services with dependencies (e.g. kafka + zookeeper or several cassandra nodes) and then adding just the files we need for each test to the rule for that test.

Depending on what your needs are you can make individual files for different tests and have a different set up for local systems, use docker-compose extends commands to split out just the containers you want, or cluster your services into logical groups then bundle them back together (we do this and have a top level command called -compose with calls out to docker-compose -f <file1> -f <file2> etc. with the passed arguments and that has worked very well.

This also lets you have separate compose files for tools to interact with your other containers via run. E.g. a psql container for interacting with your postgres container.

If these options don't work or are very cumbersome then we could look at adding this feature (or take a PR for it). The API would probably look like what you put up above, pass that through to DockerCompose and append it to all the command where it makes sense. We'd also have to deal with any commands that don't respect those options.

[bkrahmer]

We would love to have exactly what ziminer was asking for as well. In our case, we need to wait for some containers to be fully started before starting others. The easiest solution would be to start some of the containers with the rule, and then in a BeforeClass, do some logic, then reference the rule to start up an additional container. If this use case would be palatable for the maintainers, we'd be willing to work on a solution, but would ask for some guidance...

[joelea]

@bkrahmer have you looked at doing this with docker healthchecks?

You can then wire up the ordering dependencies using docker-compose.yml.

From the docs it looks like:

version: '2.1'
services:
  web:
    build: .
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_started
  redis:
    image: redis
  db:
    image: redis
    healthcheck:
      test: "exit 0"

[bkrahmer]

@joelea thanks for the tip. we will give it a try.

[bkrahmer]

We were able to solve our startup problem by simply using the healthcheck. This was a recent addition in docker-compose, added in January I believe.

[joelea]

Awesome, glad I could help and thanks for getting back with it.

Does this sound like a good solution to your problem or is there a reason it would be better in the library? I certainly prefer this solution for my projects, but I'd be happy to consider some additional library level features if there's something incomplete.

[joelea]

Closing as there's no response on the original issue and the follow up has been resolved.

[ziminer]

FYI, for anyone that sees it later, we solved the issue internally by creating a wrapper rule around DockerComposeRule with approximately the interface we suggested, built on top of the DockerComposeRule::containers method. So we have a builder like:

@Rule
ComponentCluster cluster = ComponentCluster.withServices("my-component").build();

And the ComponentCluster code does something like:

public class ComponentCluster extends ExternalResource {
    // No @Rule annotation
    DockerComposeRule rule = // create docker compose rule, pointing it at the .yml file

    @Override
    public void before() {
        ...
        Container myComponent = rule.containers().container("my-component");
        myComponent.up();
        // Do custom health check if necessary
    }

    @Override
    public void after() {
        rule.after();
    }
}

[joelea]

Thanks for the info @ziminer - do you feel like this is something that should be native to the library? My feeling so far has been that we should make basic workflows work easily out of the box by just adding an @rule annotation, and allow more complex workflows by utilising the componetnts of the library to make new custom rules (or other uses)

This is still imperfect, but seems like a better direction than trying to incorporate every workflow. Given the current usage I've seen, most workflows start simple then rapidly develop unique requirements as time goes on.

Happy to be disagreed with about this if you felt like the workaround was excessively cumbersome. I'd also love to take some suggestions about abstractions we could slide in that would've made your custom rule much easier to write.