introduce a config option to not auto-up services

[acran]

This is a PoC implementation of solution 2) from #1896:
It adds the config option auto_up as boolean to mark a service to not start by default when calling docker-compose up but instead only when given explicitly or pulled in as a dependency of another service.

The option is True by default so behavior when omitting it is the same as before. When auto_up: false is given it should only affect docker-compose up and docker-compose create (though I'm not 100% sure about any other side effects of this change).

Further discussion should take place at #1896.

Still TODO: update docs, tests and comments for the changed functions!

[moriyoshi]

I'd love to see this merged. This now conflicts with master. Please take a look into it.

[acran]

@moriyoshi I rebased my commit on the current master and so resolved the merge conflicts.
I also added updated docs for the new config option.

[chpatton013]

+1 for getting some attention on this PR from one of the repo maintainers. It's a bit of a travesty that such a simple feature doesn't exist already, let alone that the implementation has been ignored for almost a year.

[msabramo]

I would love to see this feature! Thanks, @acran for taking the time to implement this.

[drAlberT]

+1

[bgruening]

@shin- anything that needs to be done here to get it in?

[albttx]

Why this PR has been forgotten ?

[mvidalgarcia]

What is the current way of doing the same as described in this PR until it's merged?

[acran]

What is the current way of doing the same as described in this PR until it's merged?

Currently the only way to do is to outsource the definition of the optional service into its own docker-compose.yml which can the be merged/used instead with the -f flag.
See discussion in #1896.

[shin-]

Closing for the reasons highlighted in #1896 (comment) - please use a separate Compose file for services that need to be started separately.

[creynders]

This is a truly sad state of affairs. @acran 's last post nails it 100% in #1896
This is something many people want. It's a valid use case. The code's there.

By closing the comments in the other thread the demand won't go away. People will keep on requesting this. Why? Because it makes 100% sense. To everybody except you apparently @shin-

[albttx]

Vagrant has the option for the images and it's really cool to use !

https://www.vagrantup.com/docs/multi-machine/#autostart-machines

[dottgonzo]

+1

[nmclrn]

Hi @shin-

In #1896 you said "The thread is still unlocked and will continue to be as long as people refrain from being mean or impolite."

@acran's last post in #1896 does seem to sum up popular sentiment about this whole situation (source: reactions above), and that post seems neither mean nor impolite. So whilst I'd agree that popular sentiment can be misguided, I would have loved to see a reasoned rebuttal of those points (or graceful accession to the evident demand!), rather than shutting down a mostly well-intentioned discussion.

[dbjpanda]

Is auto_up option working or not ?

[acran]

Is auto_up option working or not ?

The code in this branch/PR is working, but it was never merged into the master but just closed and so is not available in upstream docker-compose unfortunately. See discussions above and in #1896.

[kitingChris]

To refuse this PR is such a poor decision.

To get you the case: We defined several services that should be started by docker-compose up.
But there are a few that need not to be started but be defined in the file.
Of course you can define those in a seperate file. But the consequence would be to have longer commands for ALL command containers.

So instead of

docker-compose composer install
docker-compose phpstan analyse
docker-compose phpunit

We need to call

docker-compose -f docker-compose-tools.yml composer install
docker-compose -f docker-compose-tools.yml phpstan analyse
docker-compose -f docker-compose-tools.yml phpunit

This is totally inconvenient. The PR would bring a huge benefit for docker-compose. Please rethink your decision @shin-

[dotchev]

Here is another use-case.

I am developing a messaging course with Apache Kafka. I selected docker-compose over Kubernets because it is much simpler to use.
In one exercise I want to start Kafka with one producer and once consumer - the simplest setup. Later on I want to start an additional consumer. It would be very natural to run just docker-compose start consumer2.
Putting the second consumer in a separate docker-compose.yml would be very awkward. Also the cleanup would not be as straightforward as docker-compose down.
Another option would be to uncomment the service in docker-compose.yml, but that would be less convenient.

[jantuitman]

Hi @shin We also have a use case.

Laravel project which sometimes does need a horizon queue for an unspecified time period and sometimes does not need. It is definately a service and not a task, as there is no defined ending moment, it rather means "this environment is providing the service" as opposed to "another environment is providing the service" so your arguments in the other PR that adding this boolean would conflate services and tasks don't make sense for our situation.

We currently use the 2 files setup, but that is cumbersome since we have an environment in which it needs to be toggled from time to time, and we cannot keep the override file from git, since it is a large file which also contains lots of other environment specific stuff. so now we have to copy/paste or comment/uncomment a service. not ideal.

So, my 2 cents: As this is such a simple boolean property I highly recommend merging this feature and dont see any disadvantages. There are far more exotic things in docker-compose than this. Thanks in advance for reading this and giving it another thought :-)

[ghost]

I've got a use case as well: I package my version of certbot/certbot to automatically renew SSL certificates of my webserver: the --webroot option to certbot certonly needs to access the nginx's document root folder to place there the http-01 acme challenge, so I use docker-compose to share a temporary volume. Something close to the following:

version: '3.7'
services:
    nginx:
        restart: always
        volumes:
            - certbot-webroot:/var/www/certbot
    my-certbot:
        volumes:
            - certbot-webroot:/var/www/certbot
volumes:
    certbot-webroot:

And I get the certificates renewed automatically and nginx to immediately use them:

  0 0 * * * docker-compose run my-certbot renew && docker-compose exec nginx nginx -s reload

Consequently I do not want this my-certbot "service" to start on any docker-compose up but only when using docker-compose run my-certbot renew from a cron job.

Finally, I don't see why adding a (non-default) option that indicates a container should not be started is a problem in itself. Sometimes getting too much opinionated comes in the way of solving real-life genuine practical user problems. Please just accept the merge request and make people happy, life is too short. Thank you.

[pdfrod]

I also have a use case for this.

The application I work on depends on multiple services (Redis, Memcached, MongoDB, etc...), however 90% of time I only need a couple of those services running to do my work. I'd prefer if by default only the essential services were launched, and the remaining ones be launched on demand. Launching every service would consume a lot of resources and slow things down unnecessarily.

I could use separate docker-compose files but that would just complicate things for no good reason.

[acran]

I updated and finalized this PR based on the most recent master. You can find it here: #7548

[timwsuqld]

Apparently we need a spec change, not just the code. See compose-spec/compose-spec#87 for an issue to add your 👍 to as well

[mkalam-alami]

Posting an update here for people coming from #1896:

It seems like the feature is making actual progress in the form of service profiles. The feature has just been merged into the Compose specification and is open for implementation at #7919 🎉

Reading through the various threads and PRs, it seems like we owe this most to @acran's patient efforts, so thank you :)

[astorath]

It seems like the feature is making actual progress in the form of service profiles. The feature has just been merged into the Compose specification and is open for implementation at #7919 🎉

This would depend on the implementation, because profiles does not specify that you should actually create disabled services - you may just skip them as well.

[tjespe]

Here is another use case:

We use docker-compose for defining which services to run and how to run them when developing our web app project locally. However, all developers might not want all services to run.

As an example: for backend developers working a lot with ElasticSearch it can be very useful to run Kibana to test stuff directly against the ElasticSearch indices, and it would be useful if the setup for this is defined in the compose file. However, a frontend developer that never works with ElasticSearch obviously would not want the Kibana service to hog the resources on their computer. The result is then that the service cannot be added to the compose file, and each backend dev needs to install and configure Kibana manually.

[acran]

@tjespe this PR was solved with the new service profiles feature released with docker-compose 1.28.0 a few weeks ago.

All the use cases here can now be implemented with that feature. Thus this PR is obsolete now.