depends_on required property error #10804

ShahimEssaid · 2023-07-12T16:44:05Z

Description

The handling of depneds_on when the referenced service is not active (due to profiles in my case) has changed. It used to ignore the missing service but now with the latest compose it is an error if the referenced service is not active. But, I noticed the new "required" property to enable the previous behavior. However, this property is causing Additional property required is not allowed

Steps To Reproduce

Environment:
Ubuntu 22.04 with Docker installed from Docker repositories.
Docker Compose version v2.19.1
Docker version 24.0.4, build 3713ee1
With example compose.yaml:

services:
  first:
    image: hello-world

  second:
    image: hello-world
    depends_on:
      first:
        condition: service_completed_successfully
        required: false

Run:
09:40 $ docker compose up
Error:
validating /home/essaids/git/gh/se/hapi-server-setup/tmp/dc1/compose.yaml: services.second.depends_on.first Additional property required is not allowed

Compose Version

Docker Compose version v2.19.1
Docker version 24.0.4, build 3713ee1

Docker Environment

Client: Docker Engine - Community
 Version:    24.0.4
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.19.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 8
  Running: 1
  Paused: 0
  Stopped: 7
 Images: 77
 Server Version: 24.0.4
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.19.0-46-generic
 Operating System: Ubuntu 22.04.2 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 6
 Total Memory: 73.4GiB
 Name: WVM4
 ID: e82f8658-d07f-477c-ab34-a5f152ef9e34
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

No response

The text was updated successfully, but these errors were encountered:

glours · 2023-07-12T16:48:40Z

👋 @ShahimEssaid
The implementation of the new depends_on.required is currently in progress and it's still not supported by Compose. But it should be available soon

ShahimEssaid · 2023-07-12T16:49:55Z

If that implementation is still in progress, why was the previous behavior not continued until then? My setup is now broken with no option to enable the previous behavior. This is not good.

To clarify, I had a setup where several services might be active or not based on profiles and the depends_on helped with ordering them as needed if they are active. Now, that setup is broken. I'll try dropping down to a previous docker/compose versions for now as a workaround.

glours · 2023-07-12T16:53:58Z

The main reason here is that we fixed the behaviour of depends_on with profiles according to the specification. You were using a bug as a feature (and this our fault not yours 😄 ), this has been discussed here
So I propose, if you're ok, to close this issue and keep the original as the main entry point

ShahimEssaid · 2023-07-12T16:57:58Z

I understand that you fixed the previous implementation but now the specification has another item that the implementation is not compliant with. So, the implementation is still not according to the spec anyway so why not keep the old behavior as well until both issues are according to the spec?

Anyway, the fix is easy for me for now. I've downgraded to Docker Compose version v2.18.1 and all is good for now. Yes, you can close but if the fix is not going to be in the next release maybe you can consider reverting the fix until both fixes are in a release.

austindrenski · 2023-07-26T23:32:30Z

@glours I just encountered this error today with a very simple project in GitHub Actions.

I'm having trouble following the who/where/why of this^ discussion, but I'm pretty sure that my case does not depend on "using a bug as a feature", so hoping we can get this reopened for some more attention on this issue.

$ docker compose config | docker buildx bake --file - --push

ERROR: failed to parse -: parsing yaml: validating : services.fetch.depends_on.otel Additional property required is not allowed, parsing hcl: -.hcl:10,73-74: Invalid character; This character is not used within the language., and 23 other diagnostic(s)

# original syntax fails
services:

  fetch:
    depends_on:
    - otel

  otel:

# partial extended syntax fails
services:

  fetch:
    depends_on:
      otel:
        condition: service_healthy

  otel:

# full extended syntax fails
services:

  fetch:
    depends_on:
      otel:
        condition: service_healthy
        required: true

  otel:

on:
  push:

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v3

    - uses: docker/setup-buildx-action@v2.9.1

    - run: docker compose config | docker buildx bake --file - --push

Docker info
  /usr/bin/docker version
  Client:
   Version:           20.10.25+azure-2
   API version:       1.41
   Go version:        go1.19.10
   Git commit:        b82b9f3a0e763304a250531cb9350aa6d93723c9
   Built:             Thu Apr  6 10:55:17 UTC 2023
   OS/Arch:           linux/amd64
   Context:           default
   Experimental:      true
  
  Server:
   Engine:
    Version:          20.10.25+azure-2
    API version:      1.41 (minimum version 1.[12]
    Go version:       go1.19.10
    Git commit:       5df983c7dbe2f8914e6efd4dd6e0083a20c41ce1
    Built:            Thu May  4 13:33:05 2023
    OS/Arch:          linux/amd64
    Experimental:     false
   containerd:
    Version:          1.6.21+azure-3
    GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
   runc:
    Version:          1.1.8
    GitCommit:        82f18fe0e44a59034f3e1f45e475fa5636e539aa
   docker-init:
    Version:          0.19.0
    GitCommit:        
  /usr/bin/docker info
  Client:
   Context:    default
   Debug Mode: false
   Plugins:
    buildx: Docker Buildx (Docker Inc., 0.11.2+azure-1)
    compose: Docker Compose (Docker Inc., 2.20.2+azure-1)
  
  Server:
   Containers: 0
    Running: 0
    Paused: 0
    Stopped: 0
   Images: 17
   Server Version: 20.10.25+azure-2
   Storage Driver: overlay2
    Backing Filesystem: extfs
    Supports d_type: true
    Native Overlay Diff: false
    userxattr: false
   Logging Driver: json-file
   Cgroup Driver: cgroupfs
   Cgroup Version: 2
   Plugins:
    Volume: local
    Network: bridge host ipvlan macvlan null overlay
    Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
   Swarm: inactive
   Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
   Default Runtime: runc
   Init Binary: docker-init
   containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
   runc version: 82f18fe0e44a59034f3e1f45e475fa5636e539aa
   init version: 
   Security Options:
    apparmor
    seccomp
     Profile: default
    cgroupns
   Kernel Version: 5.15.0-1041-azure
   Operating System: Ubuntu 22.04.2 LTS
   OSType: linux
   Architecture: x86_64
   CPUs: 2
   Total Memory: 6.769GiB
   Name: fv-az587-404
   ID: 3VSB:NXIS:24L2:M6PQ:TJO2:XRKH:EU3Y:O5W5:TBXJ:WCJP:B7JM:JHCV
   Docker Root Dir: /var/lib/docker
   Debug Mode: false
   Username: githubactions
   Registry: https://index.docker.io/v1/
   Labels:
   Experimental: false
   Insecure Registries:
    127.0.0.0/8
   Live Restore Enabled: false

Buildx version
  /usr/bin/docker buildx version
  github.com/docker/buildx 0.11.2+azure-1 9872040b66[26]

BuildKit version
  builder-4eb06548-2fb9-49d6-aa72-dbbf9e26261a0: v0.11.6

glours · 2023-07-27T08:22:56Z

Hey @austindrenski
Could just confirm you don't have any error when doing only the docker compose config command without piping wiht buildx?
The issue should be on the bake side, depends_on.required had been added in v1.17.0 of compose-go and buildx still rely on version v1.14.0
I'll prepare a PR to update the version of Compose-go in Buildx

Edit: I reproduced the issue and confirm bake won't be able to parse correctly compose file with depends_on, I'm on it

glours · 2023-07-27T08:46:47Z

The PR to bump compose-go version in buildx is open

austindrenski · 2023-07-27T13:19:33Z

Hey @austindrenski Could just confirm you don't have any error when doing only the docker compose config command without piping wiht buildx? The issue should be on the bake side, depends_on.required had been added in v1.17.0 of compose-go and buildx still rely on version v1.14.0 I'll prepare a PR to update the version of Compose-go in Buildx

Edit: I reproduced the issue and confirm bake won't be able to parse correctly compose file with depends_on, I'm on it

Thanks, @glours! That's awesome, can't tell you how much I appreciate your quick response and assist!

MAGICCC · 2023-08-02T19:57:35Z

Oh nice, I ran into the same issue as well, I will wait for the PR to get merged.

ShahimEssaid added kind/bug status/0-triage labels Jul 12, 2023

glours removed the status/0-triage label Jul 12, 2023

glours self-assigned this Jul 12, 2023

glours closed this as not planned Won't fix, can't repro, duplicate, stale Jul 12, 2023

glours mentioned this issue Jul 27, 2023

bump compose-go version to v1.17.0 to fix issue with depends_on docker/buildx#1971

Merged

yenienserrano mentioned this issue Aug 23, 2023

Add environments for 4.5 wazuh/wazuh-dashboard-plugins#5797

Merged

alexburner mentioned this issue Nov 19, 2023

[BUG] service is required by service but is disabled #10751

Closed

Vigilans mentioned this issue Apr 14, 2024

Introduce profiles attribute for depends_on to define profiles dependency compose-spec/compose-spec#488

Open

spurin mentioned this issue Oct 18, 2024

install extention occur exception spurin/diveintokubernetes#8

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

depends_on required property error #10804

depends_on required property error #10804

ShahimEssaid commented Jul 12, 2023 •

edited

Loading

glours commented Jul 12, 2023

ShahimEssaid commented Jul 12, 2023 •

edited

Loading

glours commented Jul 12, 2023 •

edited

Loading

ShahimEssaid commented Jul 12, 2023 •

edited

Loading

austindrenski commented Jul 26, 2023

glours commented Jul 27, 2023 •

edited

Loading

glours commented Jul 27, 2023

austindrenski commented Jul 27, 2023

MAGICCC commented Aug 2, 2023

depends_on required property error #10804

depends_on required property error #10804

Comments

ShahimEssaid commented Jul 12, 2023 • edited Loading

Description

Steps To Reproduce

Compose Version

Docker Environment

Anything else?

glours commented Jul 12, 2023

ShahimEssaid commented Jul 12, 2023 • edited Loading

glours commented Jul 12, 2023 • edited Loading

ShahimEssaid commented Jul 12, 2023 • edited Loading

austindrenski commented Jul 26, 2023

glours commented Jul 27, 2023 • edited Loading

glours commented Jul 27, 2023

austindrenski commented Jul 27, 2023

MAGICCC commented Aug 2, 2023

ShahimEssaid commented Jul 12, 2023 •

edited

Loading

ShahimEssaid commented Jul 12, 2023 •

edited

Loading

glours commented Jul 12, 2023 •

edited

Loading

ShahimEssaid commented Jul 12, 2023 •

edited

Loading

glours commented Jul 27, 2023 •

edited

Loading