Using labels to start a stopped container?

[beardstack]

Hey @maietta thanks for your work on this project! I may have an uncommon request - I use docker-compose and each of my stacks has a custom image that should be used to run backups (it has some tools installed that I don't want to include in the app containers for security reasons). I wasn't particularly happy with any existing backup solutions I found so far for several reasons. That custom container uses an image declared in a docker-compose.override.yml file file with labels and arguments to simplify deployments and keep things "clean" and dynamic. e.g

docker-compose.override.yml

services:
  backup:
    restart: "no"
    container_name: ${APP_SVC_NAME}_backup
    hostname: ${APP_SVC_NAME}_backup
    image: my-backup-client
    volumes:
     - store:/store
     - /etc/localtime:/etc/localtime:ro
     - app_vol_data:/var/www/app/data
     - app_vol_plugins:/var/www/app/plugins
    networks:
      - backup_net
    environment:
     - HOSTNAME=${APP_SVC_NAME}_backup
     - TZ=${TZ}
     - APP_SVC_NAME=${APP_SVC_NAME}
     - HOST_ID=${APP_SVC_NAME}
     - |
       BACKUP_DIRS=
       /var/www/app/
     - |
       EXCLUDE_FILTERS=
       *.git*
       *.tmp
    env_file:
     - .override.env
    networks:
     - backup_net
    labels:
      - chadburn.enabled=true
      - chadburn.job-exec.${APP_SVC_NAME}_backup.schedule=@daily
      - chadburn.job-exec.${APP_SVC_NAME}_backup.command=./backup.sh
      - chadburn.job-exec.${APP_SVC_NAME}_backup.container=${APP_SVC_NAME}_backup

The backup container executes and should die after first execution and start again based on the schedule. Ideally it would be great if I could add those labels to the main app declared in "${APP_SVC_NAME}" environment variable. The labels would tell the scheduler that the "${APP_SVC_NAME}_backup" sidecar should be executed at the specified intervals.

However at the moment, this wont work because it would require job-run as opposed to job-exec and my "temporary" solution is to keep the backup container running and use job-exec on it. This is not ideal (resource consumption) and I would really like to keep away from a static config file that I'd have to update manually. I've tried to use job-run from the main app labels but it doesn't register.

Is my idea possible? What would be the challenges?

[maietta]

Glad you've found some use for Chadburn, but I'm not sure I'm going to be of much help today. My brain is a bit foggy, which happens frequently these days!

Let's see if I have this right:

You have a task that needs to be performed periodically, but the target container isn't always running, hence you need to start the container to run that task.

Secondly, you'd like to pick up the app name dynamically.

If both of those ring true, I think your solution needs to be re-architected to fit the scenario. Before I propose any ideas, am I interpreting your problem correctly?

[beardstack]

Hey no stress friend, this isn't an emergency and I'm not really expecting a solution today. But yes you understand my situation.

[maietta]

Just double checking-- are you running Chadburn service elsewhere on the host, such as another compose, or using the binary? Because the Chadburn service doesn't appear in what you provided. (just asking, in case it somehow got overlooked)

The Chadburn service should always be running, so a restart: "always" should be used.

As far as I can tell, your docker-compose stack looks good so far, but only missing the Chadburn service, like so:

services:
  chadburn:
    image: premoweb/chadburn:latest
   restart: "always"
    depends_on:
      - backup
    command: daemon
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    networks:
     - backup_net

  backup:
    restart: "no"
    container_name: ${APP_SVC_NAME}_backup
    hostname: ${APP_SVC_NAME}_backup
    image: my-backup-client
    volumes:
     - store:/store
     - /etc/localtime:/etc/localtime:ro
     - app_vol_data:/var/www/app/data
     - app_vol_plugins:/var/www/app/plugins
    networks:
      - backup_net
    environment:
     - HOSTNAME=${APP_SVC_NAME}_backup
     - TZ=${TZ}
     - APP_SVC_NAME=${APP_SVC_NAME}
     - HOST_ID=${APP_SVC_NAME}
     - |
       BACKUP_DIRS=
       /var/www/app/
     - |
       EXCLUDE_FILTERS=
       *.git*
       *.tmp
    env_file:
     - .override.env
    networks:
     - backup_net
    labels:
      - chadburn.enabled=true
      - chadburn.job-exec.${APP_SVC_NAME}_backup.schedule=@daily
      - chadburn.job-exec.${APP_SVC_NAME}_backup.command=./backup.sh
      - chadburn.job-exec.${APP_SVC_NAME}_backup.container=${APP_SVC_NAME}_backup

Just a couple of notes:

• Chadburn can't wake up and run your application stack's backup service directly without some other wrapper, such as involving a script that runs your backup job. It's designed to work directly on existing containers or invoke containers directly. It wasn't designed to invoke a composable service directly.
• I don't know if Chadburn needs a "depends_on" option but suspect you may need to let Chadburn access the backup_net network.

[beardstack]

To clarify Chadburn runs in it's own docker compose stack and is always running. I use namespaces & separate networks for different things for security reasons. I don't think that the "composable" service matters (I'm not really sure tbh) - Since I standardize container naming, i can run backup.sh directly when using docker exec. Networks don't really matter in this particular case.

The docker compose file already executes the backup container on start, once done executing, it should just stop and be in an exited stated, the container is still there and can just be run again using docker run

[beardstack]

I was thinking of a "hack" but I'm not familiar enough with go to implement it without some guidance. How could I implement a new label modifier like job-revive

for instance for each service I wanted to backup, I would add the following labels

    labels:
      - chadburn.enabled=true
      - chadburn.job-revive.${APP_SVC_NAME}.schedule=@daily
      - chadburn.job-revive.${APP_SVC_NAME}.command=./backup.sh
      - chadburn.job-revive.${APP_SVC_NAME}.container=my-container

And while that service is running chadburn would docker run my-container

[beardstack]

hey @maietta I've been able to patch the code to allow job-run labels to work for any running container. You can see my changes in this commit I'm quite inexperienced in golang and this code most certainly can be optimized, however I believe that the additions provide valuable functionality to run one off or scheduled 'sidecar' style containers (backup, log rotation etc). I am not sure why the job-run functionality was limited to the chadburn container initially, but it's definitely possible and useful to remove that limitation. I'd like to hear your thoughts and see if it's possible to merge those changes in the main code.