Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[0.13] bake: changed behavior in output configuration #2325

Closed
3 tasks done
davhdavh opened this issue Mar 8, 2024 · 9 comments · Fixed by #2330
Closed
3 tasks done

[0.13] bake: changed behavior in output configuration #2325

davhdavh opened this issue Mar 8, 2024 · 9 comments · Fixed by #2330
Labels
area/bake kind/bug Something isn't working
Milestone

Comments

@davhdavh
Copy link

davhdavh commented Mar 8, 2024

Contributing guidelines

I've found a bug and checked that ...

  • ... the documentation does not mention anything about my problem
  • ... there are no open or closed issues that are related to my problem

Description

It seems some logic changed in merging bake files.

Expected behaviour

upgrade from 0.12.1 to 0.13 didn't break build

Actual behaviour

upgrade from 0.12.1 to 0.13 didn't break build.

v0.12.1:

  {
    "group": {
      "default": {
        "targets": [
          "a",
          "b",
          "c",
          "d",
          "test"
        ]
      }
    },
    "target": {
      "build-base": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/X/X/actions/runs/8198168165"
        ],
        "context": ".",
        "dockerfile": "Dockerfile.build-base.Dockerfile",
        "platforms": [
          "linux/amd64"
        ],
        "pull": true
      },
      "p": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/X/X/actions/runs/8198168165"
        ],
        "context": "Z/Z",
        "dockerfile": "Dockerfile",
        "pull": true
      },
      "b": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/X/X/actions/runs/8198168165"
        ],
        "context": ".",
        "contexts": {
          "builder": "target:b-builder",
          "p": "target:p",
          "e": "target:e",
          "runtime-base": "target:runtime-base"
        },
        "dockerfile": "B/Dockerfile.deploy.Dockerfile",
        "args": {
          "DOCKER_META_IMAGES": "***/***",
          "DOCKER_META_VERSION": "b-1176",
          "PROJ": "B"
        },
        "labels": {
          "org.opencontainers.image.created": "2024-03-08T03:10:43.389Z",
          "org.opencontainers.image.description": "X B",
          "org.opencontainers.image.licenses": "",
          "org.opencontainers.image.revision": "8ec5766dba6025b8fd13d585e80abbb217c1862c",
          "org.opencontainers.image.source": "https://github.com/X/X",
          "org.opencontainers.image.title": "X B",
          "org.opencontainers.image.url": "https://github.com/X/X",
          "org.opencontainers.image.vendor": "X - Q A/S",
          "org.opencontainers.image.version": "b-1176"
        },
        "tags": [
          "***/***:b-1176"
        ],
        "platforms": [
          "linux/amd64"
        ],
        "output": [
          "type=image,push=true,push=true,push=true"
        ],
        "pull": true
      },
      "b-builder": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/X/X/actions/runs/8198168165"
        ],
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "B"
        },
        "platforms": [
          "linux/amd64"
        ],
        "pull": true
      },
      "d": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/X/X/actions/runs/8198168165"
        ],
        "context": ".",
        "contexts": {
          "builder": "target:d-builder",
          "runtime-base": "target:runtime-base"
        },
        "dockerfile": "Dockerfile.all.Dockerfile",
        "args": {
          "DOCKER_META_IMAGES": "***/***",
          "DOCKER_META_VERSION": "d-1176",
          "PROJ": "X.D"
        },
        "labels": {
          "org.opencontainers.image.created": "2024-03-08T03:10:44.747Z",
          "org.opencontainers.image.description": "X D",
          "org.opencontainers.image.licenses": "",
          "org.opencontainers.image.revision": "8ec5766dba6025b8fd13d585e80abbb217c1862c",
          "org.opencontainers.image.source": "https://github.com/X/X",
          "org.opencontainers.image.title": "X D",
          "org.opencontainers.image.url": "https://github.com/X/X",
          "org.opencontainers.image.vendor": "X - Q A/S",
          "org.opencontainers.image.version": "d-1176"
        },
        "tags": [
          "***/***:d-1176"
        ],
        "platforms": [
          "linux/amd64"
        ],
        "output": [
          "type=image,push=true,push=true,push=true"
        ],
        "pull": true
      },
      "d-builder": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/X/X/actions/runs/8198168165"
        ],
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "X.D"
        },
        "platforms": [
          "linux/amd64"
        ],
        "pull": true
      },
      "e": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/X/X/actions/runs/8198168165"
        ],
        "context": "Y/source",
        "dockerfile": "Dockerfile",
        "pull": true
      },
      "c": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/X/X/actions/runs/8198168165"
        ],
        "context": ".",
        "contexts": {
          "builder": "target:c-builder",
          "runtime-base": "target:runtime-base"
        },
        "dockerfile": "Dockerfile.all.Dockerfile",
        "args": {
          "DOCKER_META_IMAGES": "***/***",
          "DOCKER_META_VERSION": "c-1176",
          "PROJ": "X.C"
        },
        "labels": {
          "org.opencontainers.image.created": "2024-03-08T03:10:47.377Z",
          "org.opencontainers.image.description": "X c",
          "org.opencontainers.image.licenses": "",
          "org.opencontainers.image.revision": "8ec5766dba6025b8fd13d585e80abbb217c1862c",
          "org.opencontainers.image.source": "https://github.com/X/X",
          "org.opencontainers.image.title": "X c",
          "org.opencontainers.image.url": "https://github.com/X/X",
          "org.opencontainers.image.vendor": "X - Q A/S",
          "org.opencontainers.image.version": "c-1176"
        },
        "tags": [
          "***/***:c-1176"
        ],
        "platforms": [
          "linux/amd64"
        ],
        "output": [
          "type=image,push=true,push=true,push=true"
        ],
        "pull": true
      },
      "c-builder": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/X/X/actions/runs/8198168165"
        ],
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "X.C"
        },
        "platforms": [
          "linux/amd64"
        ],
        "pull": true
      },
      "runtime-base": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/X/X/actions/runs/8198168165"
        ],
        "context": ".",
        "dockerfile": "Dockerfile.runtime-base.Dockerfile",
        "platforms": [
          "linux/amd64"
        ],
        "pull": true
      },
      "a": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/X/X/actions/runs/8198168165"
        ],
        "context": ".",
        "contexts": {
          "builder": "target:a-builder",
          "runtime-base": "target:runtime-base"
        },
        "dockerfile": "Dockerfile.all.Dockerfile",
        "args": {
          "DOCKER_META_IMAGES": "***/***",
          "DOCKER_META_VERSION": "a-1176",
          "PROJ": "X.A"
        },
        "labels": {
          "org.opencontainers.image.created": "2024-03-08T03:10:46.039Z",
          "org.opencontainers.image.description": "X a",
          "org.opencontainers.image.licenses": "",
          "org.opencontainers.image.revision": "8ec5766dba6025b8fd13d585e80abbb217c1862c",
          "org.opencontainers.image.source": "https://github.com/X/X",
          "org.opencontainers.image.title": "X a",
          "org.opencontainers.image.url": "https://github.com/X/X",
          "org.opencontainers.image.vendor": "X - Q A/S",
          "org.opencontainers.image.version": "a-1176"
        },
        "tags": [
          "***/***:a-1176"
        ],
        "platforms": [
          "linux/amd64"
        ],
        "output": [
          "type=image,push=true,push=true,push=true"
        ],
        "pull": true
      },
      "a-builder": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/X/X/actions/runs/8198168165"
        ],
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "X.A"
        },
        "platforms": [
          "linux/amd64"
        ],
        "pull": true
      },
      "test": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/X/X/actions/runs/8198168165"
        ],
        "context": ".",
        "contexts": {
          "builder": "target:test-builder",
          "test-base": "target:test-base"
        },
        "dockerfile": "Dockerfile.all-test.Dockerfile",
        "platforms": [
          "linux/amd64"
        ],
        "output": [
          "type=local,dest=./test-results-output"
        ],
        "pull": true
      },
      "test-base": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/X/X/actions/runs/8198168165"
        ],
        "context": ".",
        "dockerfile": "Dockerfile.test-base.Dockerfile",
        "platforms": [
          "linux/amd64"
        ],
        "pull": true
      },
      "test-builder": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/X/X/actions/runs/8198168165"
        ],
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "X.Test"
        },
        "platforms": [
          "linux/amd64"
        ],
        "pull": true
      }
    }
  }

Final merged bake file:

0.13.0:

   {
    "group": {
      "default": {
        "targets": [
          "a",
          "b",
          "c",
          "d",
          "test"
        ]
      }
    },
    "target": {
      "build-base": {
        "context": ".",
        "dockerfile": "Dockerfile.build-base.Dockerfile",
        "platforms": [
          "linux/amd64"
        ],
        "pull": true
      },
      "p": {
        "context": "Z/Z",
        "dockerfile": "Dockerfile",
        "pull": true
      },
      "b": {
        "context": ".",
        "contexts": {
          "builder": "target:b-builder",
          "p": "target:p",
          "e": "target:e",
          "runtime-base": "target:runtime-base"
        },
        "dockerfile": "B/Dockerfile.deploy.Dockerfile",
        "args": {
          "DOCKER_META_IMAGES": "***/***",
          "DOCKER_META_VERSION": "b-1176",
          "PROJ": "B"
        },
        "labels": {
          "org.opencontainers.image.created": "2024-03-08T03:32:51.778Z",
          "org.opencontainers.image.description": "X B",
          "org.opencontainers.image.licenses": "",
          "org.opencontainers.image.revision": "8ec5766dba6025b8fd13d585e80abbb217c1862c",
          "org.opencontainers.image.source": "https://github.com/X/X",
          "org.opencontainers.image.title": "X B",
          "org.opencontainers.image.url": "https://github.com/X/X",
          "org.opencontainers.image.vendor": "X - Q A/S",
          "org.opencontainers.image.version": "b-1176"
        },
        "tags": [
          "***/***:b-1176"
        ],
        "platforms": [
          "linux/amd64"
        ],
        "output": [
          "type=registry"
        ],
        "pull": true
      },
      "b-builder": {
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "B"
        },
        "platforms": [
          "linux/amd64"
        ],
        "pull": true
      },
      "d": {
        "context": ".",
        "contexts": {
          "builder": "target:d-builder",
          "runtime-base": "target:runtime-base"
        },
        "dockerfile": "Dockerfile.all.Dockerfile",
        "args": {
          "DOCKER_META_IMAGES": "***/***",
          "DOCKER_META_VERSION": "d-1176",
          "PROJ": "X.D"
        },
        "labels": {
          "org.opencontainers.image.created": "2024-03-08T03:32:53.051Z",
          "org.opencontainers.image.description": "X D",
          "org.opencontainers.image.licenses": "",
          "org.opencontainers.image.revision": "8ec5766dba6025b8fd13d585e80abbb217c1862c",
          "org.opencontainers.image.source": "https://github.com/X/X",
          "org.opencontainers.image.title": "X D",
          "org.opencontainers.image.url": "https://github.com/X/X",
          "org.opencontainers.image.vendor": "X - Q A/S",
          "org.opencontainers.image.version": "d-1176"
        },
        "tags": [
          "***/***:d-1176"
        ],
        "platforms": [
          "linux/amd64"
        ],
        "output": [
          "type=registry"
        ],
        "pull": true
      },
      "d-builder": {
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "X.D"
        },
        "platforms": [
          "linux/amd64"
        ],
        "pull": true
      },
      "e": {
        "context": "Y/source",
        "dockerfile": "Dockerfile",
        "pull": true
      },
      "c": {
        "context": ".",
        "contexts": {
          "builder": "target:c-builder",
          "runtime-base": "target:runtime-base"
        },
        "dockerfile": "Dockerfile.all.Dockerfile",
        "args": {
          "DOCKER_META_IMAGES": "***/***",
          "DOCKER_META_VERSION": "c-1176",
          "PROJ": "X.C"
        },
        "labels": {
          "org.opencontainers.image.created": "2024-03-08T03:32:55.763Z",
          "org.opencontainers.image.description": "X c",
          "org.opencontainers.image.licenses": "",
          "org.opencontainers.image.revision": "8ec5766dba6025b8fd13d585e80abbb217c1862c",
          "org.opencontainers.image.source": "https://github.com/X/X",
          "org.opencontainers.image.title": "X c",
          "org.opencontainers.image.url": "https://github.com/X/X",
          "org.opencontainers.image.vendor": "X - Q A/S",
          "org.opencontainers.image.version": "c-1176"
        },
        "tags": [
          "***/***:c-1176"
        ],
        "platforms": [
          "linux/amd64"
        ],
        "output": [
          "type=registry"
        ],
        "pull": true
      },
      "c-builder": {
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "X.C"
        },
        "platforms": [
          "linux/amd64"
        ],
        "pull": true
      },
      "runtime-base": {
        "context": ".",
        "dockerfile": "Dockerfile.runtime-base.Dockerfile",
        "platforms": [
          "linux/amd64"
        ],
        "pull": true
      },
      "a": {
        "context": ".",
        "contexts": {
          "builder": "target:a-builder",
          "runtime-base": "target:runtime-base"
        },
        "dockerfile": "Dockerfile.all.Dockerfile",
        "args": {
          "DOCKER_META_IMAGES": "***/***",
          "DOCKER_META_VERSION": "a-1176",
          "PROJ": "X.A"
        },
        "labels": {
          "org.opencontainers.image.created": "2024-03-08T03:32:54.300Z",
          "org.opencontainers.image.description": "X a",
          "org.opencontainers.image.licenses": "",
          "org.opencontainers.image.revision": "8ec5766dba6025b8fd13d585e80abbb217c1862c",
          "org.opencontainers.image.source": "https://github.com/X/X",
          "org.opencontainers.image.title": "X a",
          "org.opencontainers.image.url": "https://github.com/X/X",
          "org.opencontainers.image.vendor": "X - Q A/S",
          "org.opencontainers.image.version": "a-1176"
        },
        "tags": [
          "***/***:a-1176"
        ],
        "platforms": [
          "linux/amd64"
        ],
        "output": [
          "type=registry"
        ],
        "pull": true
      },
      "a-builder": {
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "X.A"
        },
        "platforms": [
          "linux/amd64"
        ],
        "pull": true
      },
      "test": {
        "context": ".",
        "contexts": {
          "builder": "target:test-builder",
          "test-base": "target:test-base"
        },
        "dockerfile": "Dockerfile.all-test.Dockerfile",
        "platforms": [
          "linux/amd64"
        ],
        "output": [
          "type=registry"
        ],
        "pull": true
      },
      "test-base": {
        "context": ".",
        "dockerfile": "Dockerfile.test-base.Dockerfile",
        "platforms": [
          "linux/amd64"
        ],
        "pull": true
      },
      "test-builder": {
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "X.Test"
        },
        "platforms": [
          "linux/amd64"
        ],
        "pull": true
      }
    }
  }

Summary

0.12.1 preserved the original files test.output: "type=local,dest=./test-results-output", 0.13 overwrites it to "type=registry".
0.12.1 adds attest, 0.13 doesn't.

I assume the error message comes from the test.output being update from local to registry.

Buildx version

0.13.0

Docker info

No response

Builders list

not relevant

Configuration

github

    - name: Build
      uses: docker/bake-action@511fde2517761e303af548ec9e0ea74a8a100112 # v4
      with:
        files: |
          ./docker-bake.json
          ${{ steps.meta-a.outputs.bake-file }}
          ${{ steps.meta-b.outputs.bake-file }}
          ${{ steps.meta-c.outputs.bake-file }}
          ${{ steps.meta-d.outputs.bake-file }}
        targets: default
        push: true
        pull: true

docker-bake.json:

{
  "group": {
    "default": {
      "targets": [
        "a",
        "b",
        "c",
        "d",
        "test"
      ]
    }
  },
  "target": {
    "c-tags": {},
    "c": {
      "inherits": [ "c-tags", "defaults" ],
      "contexts": {
        "builder": "target:c-builder"
      },
      "args": {
        "PROJ": "X.C"
      }
    },
    "c-builder": {
      "inherits": [ "builder" ],
      "args": {
        "PROJ": "X.C"
      }
    },
    "d-tags": {},
    "d": {
      "inherits": [ "d-tags", "defaults" ],
      "contexts": {
        "builder": "target:d-builder"
      },
      "args": {
        "PROJ": "X.D"
      }
    },
    "d-builder": {
      "inherits": [ "builder" ],
      "args": {
        "PROJ": "X.D"
      }
    },
    "a-tags": {},
    "a": {
      "inherits": [ "a-tags", "defaults" ],
      "contexts": {
        "builder": "target:a-builder"
      },
      "args": {
        "PROJ": "X.A"
      }
    },
    "a-builder": {
      "inherits": [ "builder" ],
      "args": {
        "PROJ": "X.A"
      }
    },
    "b-tags": {},
    "b": {
      "inherits": [ "b-tags", "defaults" ],
      "args": {
        "PROJ": "B"
      },
      "dockerfile": "B/Dockerfile.deploy.Dockerfile",
      "contexts": {
        "e": "target:e",
        "p": "target:p",
        "builder": "target:b-builder"
      }
    },
    "b-builder": {
      "inherits": [ "builder" ],
      "args": {
        "PROJ": "B"
      }
    },
    "e": {
      "context": "Y/source",
      "dockerfile": "Dockerfile"
    },
    "p": {
      "context": "Z/Z",
      "dockerfile": "Dockerfile"
    },

    "test": {
      "inherits": [ "all_defaults" ],
      "dockerfile": "Dockerfile.all-test.Dockerfile",
      "contexts": {
        "test-base": "target:test-base",
        "builder": "target:test-builder"
      },
      "output": [ "type=local,dest=./test-results-output" ]
    },
    "test-builder": {
      "inherits": [ "builder" ],
      "args": {
        "PROJ": "X.Test"
      }
    },
    "all_defaults": {
      "platforms": [ "linux/amd64" ],
      "context": "."
    },
    "defaults": {
      "inherits": [ "all_defaults" ],
      "dockerfile": "Dockerfile.all.Dockerfile",
      "contexts": {
        "runtime-base": "target:runtime-base"
      }
    },
    "builder": {
      "inherits": [ "all_defaults" ],
      "dockerfile": "Dockerfile.builder.Dockerfile",
      "contexts": {
        "build-base": "target:build-base"
      }
    },
    "runtime-base": {
      "inherits": [ "all_defaults" ],
      "dockerfile": "Dockerfile.runtime-base.Dockerfile"
    },
    "build-base": {
      "inherits": [ "all_defaults" ],
      "dockerfile": "Dockerfile.build-base.Dockerfile"
    },
    "test-base": {
      "inherits": [ "all_defaults" ],
      "dockerfile": "Dockerfile.test-base.Dockerfile"
    }
  }
}

steps.meta-a.outputs.bake-file :

  {
    "tags": [
      "***/***:a-1176"
    ],
    "labels": {
      "org.opencontainers.image.created": "2024-03-08T03:32:55.763Z",
      "org.opencontainers.image.description": "X a",
      "org.opencontainers.image.licenses": "",
      "org.opencontainers.image.revision": "8ec5766dba6025b8fd13d585e80abbb217c1862c",
      "org.opencontainers.image.source": "https://github.com/X/X",
      "org.opencontainers.image.title": "X a",
      "org.opencontainers.image.url": "https://github.com/X/X",
      "org.opencontainers.image.vendor": "X - Q A/S",
      "org.opencontainers.image.version": "a-1176"
    },
    "annotations": [
      "manifest:org.opencontainers.image.created=2024-03-08T03:32:55.763Z",
      "manifest:org.opencontainers.image.description=",
      "manifest:org.opencontainers.image.licenses=",
      "manifest:org.opencontainers.image.revision=8ec5766dba6025b8fd13d585e80abbb217c1862c",
      "manifest:org.opencontainers.image.source=https://github.com/X/X",
      "manifest:org.opencontainers.image.title=X",
      "manifest:org.opencontainers.image.url=https://github.com/X/X",
      "manifest:org.opencontainers.image.version=a-1176"
    ]
  }

Build logs

No response

Additional info

No response

@davhdavh
Copy link
Author

davhdavh commented Mar 8, 2024

I think this is the cause #2302

@crazy-max
Copy link
Member

crazy-max commented Mar 8, 2024

I don't think the following step from your workflow matches the output of merged files:

    - name: Build
      uses: docker/bake-action@511fde2517761e303af548ec9e0ea74a8a100112 # v4
      with:
        files: |
          ./docker-bake.json
          ${{ steps.meta-a.outputs.bake-file }}
          ${{ steps.meta-b.outputs.bake-file }}
          ${{ steps.meta-c.outputs.bake-file }}
          ${{ steps.meta-d.outputs.bake-file }}
        targets: default

Can't repro with either v0.12.1 or v0.13.0 for output field but repro missing provenance in bake-action:

2325.bake.json

Show
{
  "group": {
    "default": {
      "targets": [
        "a",
        "b",
        "c",
        "d",
        "test"
      ]
    }
  },
  "target": {
    "c-tags": {},
    "c": {
      "inherits": [ "c-tags", "defaults" ],
      "contexts": {
        "builder": "target:c-builder"
      },
      "args": {
        "PROJ": "X.C"
      }
    },
    "c-builder": {
      "inherits": [ "builder" ],
      "args": {
        "PROJ": "X.C"
      }
    },
    "d-tags": {},
    "d": {
      "inherits": [ "d-tags", "defaults" ],
      "contexts": {
        "builder": "target:d-builder"
      },
      "args": {
        "PROJ": "X.D"
      }
    },
    "d-builder": {
      "inherits": [ "builder" ],
      "args": {
        "PROJ": "X.D"
      }
    },
    "a-tags": {},
    "a": {
      "inherits": [ "a-tags", "defaults" ],
      "contexts": {
        "builder": "target:a-builder"
      },
      "args": {
        "PROJ": "X.A"
      }
    },
    "a-builder": {
      "inherits": [ "builder" ],
      "args": {
        "PROJ": "X.A"
      }
    },
    "b-tags": {},
    "b": {
      "inherits": [ "b-tags", "defaults" ],
      "args": {
        "PROJ": "B"
      },
      "dockerfile": "B/Dockerfile.deploy.Dockerfile",
      "contexts": {
        "e": "target:e",
        "p": "target:p",
        "builder": "target:b-builder"
      }
    },
    "b-builder": {
      "inherits": [ "builder" ],
      "args": {
        "PROJ": "B"
      }
    },
    "e": {
      "context": "Y/source",
      "dockerfile": "Dockerfile"
    },
    "p": {
      "context": "Z/Z",
      "dockerfile": "Dockerfile"
    },

    "test": {
      "inherits": [ "all_defaults" ],
      "dockerfile": "Dockerfile.all-test.Dockerfile",
      "contexts": {
        "test-base": "target:test-base",
        "builder": "target:test-builder"
      },
      "output": [ "type=local,dest=./test-results-output" ]
    },
    "test-builder": {
      "inherits": [ "builder" ],
      "args": {
        "PROJ": "X.Test"
      }
    },
    "all_defaults": {
      "platforms": [ "linux/amd64" ],
      "context": "."
    },
    "defaults": {
      "inherits": [ "all_defaults" ],
      "dockerfile": "Dockerfile.all.Dockerfile",
      "contexts": {
        "runtime-base": "target:runtime-base"
      }
    },
    "builder": {
      "inherits": [ "all_defaults" ],
      "dockerfile": "Dockerfile.builder.Dockerfile",
      "contexts": {
        "build-base": "target:build-base"
      }
    },
    "runtime-base": {
      "inherits": [ "all_defaults" ],
      "dockerfile": "Dockerfile.runtime-base.Dockerfile"
    },
    "build-base": {
      "inherits": [ "all_defaults" ],
      "dockerfile": "Dockerfile.build-base.Dockerfile"
    },
    "test-base": {
      "inherits": [ "all_defaults" ],
      "dockerfile": "Dockerfile.test-base.Dockerfile"
    }
  }
}

2325.a.json

Show
{
  "tags": [
    "foo/bar:a-1176"
  ],
  "labels": {
    "org.opencontainers.image.created": "2024-03-08T03:32:55.763Z",
    "org.opencontainers.image.description": "X a",
    "org.opencontainers.image.licenses": "",
    "org.opencontainers.image.revision": "8ec5766dba6025b8fd13d585e80abbb217c1862c",
    "org.opencontainers.image.source": "https://github.com/X/X",
    "org.opencontainers.image.title": "X a",
    "org.opencontainers.image.url": "https://github.com/X/X",
    "org.opencontainers.image.vendor": "X - Q A/S",
    "org.opencontainers.image.version": "a-1176"
  },
  "annotations": [
    "manifest:org.opencontainers.image.created=2024-03-08T03:32:55.763Z",
    "manifest:org.opencontainers.image.description=",
    "manifest:org.opencontainers.image.licenses=",
    "manifest:org.opencontainers.image.revision=8ec5766dba6025b8fd13d585e80abbb217c1862c",
    "manifest:org.opencontainers.image.source=https://github.com/X/X",
    "manifest:org.opencontainers.image.title=X",
    "manifest:org.opencontainers.image.url=https://github.com/X/X",
    "manifest:org.opencontainers.image.version=a-1176"
  ]
}

merge result with 0.12.1:

Show
  {
    "group": {
      "default": {
        "targets": [
          "a",
          "b",
          "c",
          "d",
          "test"
        ]
      }
    },
    "target": {
      "a": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/docker/test-docker-action/actions/runs/8201804346"
        ],
        "context": ".",
        "contexts": {
          "builder": "target:a-builder",
          "runtime-base": "target:runtime-base"
        },
        "dockerfile": "Dockerfile.all.Dockerfile",
        "args": {
          "PROJ": "X.A"
        },
        "platforms": [
          "linux/amd64"
        ]
      },
      "a-builder": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/docker/test-docker-action/actions/runs/8201804346"
        ],
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "X.A"
        },
        "platforms": [
          "linux/amd64"
        ]
      },
      "b": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/docker/test-docker-action/actions/runs/8201804346"
        ],
        "context": ".",
        "contexts": {
          "builder": "target:b-builder",
          "e": "target:e",
          "p": "target:p",
          "runtime-base": "target:runtime-base"
        },
        "dockerfile": "B/Dockerfile.deploy.Dockerfile",
        "args": {
          "PROJ": "B"
        },
        "platforms": [
          "linux/amd64"
        ]
      },
      "b-builder": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/docker/test-docker-action/actions/runs/8201804346"
        ],
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "B"
        },
        "platforms": [
          "linux/amd64"
        ]
      },
      "build-base": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/docker/test-docker-action/actions/runs/8201804346"
        ],
        "context": ".",
        "dockerfile": "Dockerfile.build-base.Dockerfile",
        "platforms": [
          "linux/amd64"
        ]
      },
      "c": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/docker/test-docker-action/actions/runs/8201804346"
        ],
        "context": ".",
        "contexts": {
          "builder": "target:c-builder",
          "runtime-base": "target:runtime-base"
        },
        "dockerfile": "Dockerfile.all.Dockerfile",
        "args": {
          "PROJ": "X.C"
        },
        "platforms": [
          "linux/amd64"
        ]
      },
      "c-builder": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/docker/test-docker-action/actions/runs/8201804346"
        ],
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "X.C"
        },
        "platforms": [
          "linux/amd64"
        ]
      },
      "d": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/docker/test-docker-action/actions/runs/8201804346"
        ],
        "context": ".",
        "contexts": {
          "builder": "target:d-builder",
          "runtime-base": "target:runtime-base"
        },
        "dockerfile": "Dockerfile.all.Dockerfile",
        "args": {
          "PROJ": "X.D"
        },
        "platforms": [
          "linux/amd64"
        ]
      },
      "d-builder": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/docker/test-docker-action/actions/runs/8201804346"
        ],
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "X.D"
        },
        "platforms": [
          "linux/amd64"
        ]
      },
      "e": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/docker/test-docker-action/actions/runs/8201804346"
        ],
        "context": "Y/source",
        "dockerfile": "Dockerfile"
      },
      "p": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/docker/test-docker-action/actions/runs/8201804346"
        ],
        "context": "Z/Z",
        "dockerfile": "Dockerfile"
      },
      "runtime-base": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/docker/test-docker-action/actions/runs/8201804346"
        ],
        "context": ".",
        "dockerfile": "Dockerfile.runtime-base.Dockerfile",
        "platforms": [
          "linux/amd64"
        ]
      },
      "test": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/docker/test-docker-action/actions/runs/8201804346"
        ],
        "context": ".",
        "contexts": {
          "builder": "target:test-builder",
          "test-base": "target:test-base"
        },
        "dockerfile": "Dockerfile.all-test.Dockerfile",
        "platforms": [
          "linux/amd64"
        ],
        "output": [
          "type=local,dest=./test-results-output"
        ]
      },
      "test-base": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/docker/test-docker-action/actions/runs/8201804346"
        ],
        "context": ".",
        "dockerfile": "Dockerfile.test-base.Dockerfile",
        "platforms": [
          "linux/amd64"
        ]
      },
      "test-builder": {
        "attest": [
          "type=provenance,mode=min,inline-only=true,builder-id=https://github.com/docker/test-docker-action/actions/runs/8201804346"
        ],
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "X.Test"
        },
        "platforms": [
          "linux/amd64"
        ]
      }
    }
  }

merge result with 0.13.0:

Show
  {
    "group": {
      "default": {
        "targets": [
          "a",
          "b",
          "c",
          "d",
          "test"
        ]
      }
    },
    "target": {
      "a": {
        "context": ".",
        "contexts": {
          "builder": "target:a-builder",
          "runtime-base": "target:runtime-base"
        },
        "dockerfile": "Dockerfile.all.Dockerfile",
        "args": {
          "PROJ": "X.A"
        },
        "platforms": [
          "linux/amd64"
        ]
      },
      "a-builder": {
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "X.A"
        },
        "platforms": [
          "linux/amd64"
        ]
      },
      "b": {
        "context": ".",
        "contexts": {
          "builder": "target:b-builder",
          "e": "target:e",
          "p": "target:p",
          "runtime-base": "target:runtime-base"
        },
        "dockerfile": "B/Dockerfile.deploy.Dockerfile",
        "args": {
          "PROJ": "B"
        },
        "platforms": [
          "linux/amd64"
        ]
      },
      "b-builder": {
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "B"
        },
        "platforms": [
          "linux/amd64"
        ]
      },
      "build-base": {
        "context": ".",
        "dockerfile": "Dockerfile.build-base.Dockerfile",
        "platforms": [
          "linux/amd64"
        ]
      },
      "c": {
        "context": ".",
        "contexts": {
          "builder": "target:c-builder",
          "runtime-base": "target:runtime-base"
        },
        "dockerfile": "Dockerfile.all.Dockerfile",
        "args": {
          "PROJ": "X.C"
        },
        "platforms": [
          "linux/amd64"
        ]
      },
      "c-builder": {
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "X.C"
        },
        "platforms": [
          "linux/amd64"
        ]
      },
      "d": {
        "context": ".",
        "contexts": {
          "builder": "target:d-builder",
          "runtime-base": "target:runtime-base"
        },
        "dockerfile": "Dockerfile.all.Dockerfile",
        "args": {
          "PROJ": "X.D"
        },
        "platforms": [
          "linux/amd64"
        ]
      },
      "d-builder": {
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "X.D"
        },
        "platforms": [
          "linux/amd64"
        ]
      },
      "e": {
        "context": "Y/source",
        "dockerfile": "Dockerfile"
      },
      "p": {
        "context": "Z/Z",
        "dockerfile": "Dockerfile"
      },
      "runtime-base": {
        "context": ".",
        "dockerfile": "Dockerfile.runtime-base.Dockerfile",
        "platforms": [
          "linux/amd64"
        ]
      },
      "test": {
        "context": ".",
        "contexts": {
          "builder": "target:test-builder",
          "test-base": "target:test-base"
        },
        "dockerfile": "Dockerfile.all-test.Dockerfile",
        "platforms": [
          "linux/amd64"
        ],
        "output": [
          "type=local,dest=./test-results-output"
        ]
      },
      "test-base": {
        "context": ".",
        "dockerfile": "Dockerfile.test-base.Dockerfile",
        "platforms": [
          "linux/amd64"
        ]
      },
      "test-builder": {
        "context": ".",
        "contexts": {
          "build-base": "target:build-base"
        },
        "dockerfile": "Dockerfile.builder.Dockerfile",
        "args": {
          "PROJ": "X.Test"
        },
        "platforms": [
          "linux/amd64"
        ]
      }
    }
  }

I don't repro locally either the output with:

Show
$ docker buildx bake -f 2325.bake.json -f 2325.a.json default --print
#1 [internal] load local bake definitions
#1 reading 2325.bake.json 2.82kB / 2.82kB done
#1 reading 2325.a.json 1.12kB / 1.12kB done
#1 DONE 0.0s
{
  "group": {
    "default": {
      "targets": [
        "a",
        "b",
        "c",
        "d",
        "test"
      ]
    }
  },
  "target": {
    "a": {
      "context": ".",
      "contexts": {
        "builder": "target:a-builder",
        "runtime-base": "target:runtime-base"
      },
      "dockerfile": "Dockerfile.all.Dockerfile",
      "args": {
        "PROJ": "X.A"
      },
      "platforms": [
        "linux/amd64"
      ]
    },
    "a-builder": {
      "context": ".",
      "contexts": {
        "build-base": "target:build-base"
      },
      "dockerfile": "Dockerfile.builder.Dockerfile",
      "args": {
        "PROJ": "X.A"
      },
      "platforms": [
        "linux/amd64"
      ]
    },
    "b": {
      "context": ".",
      "contexts": {
        "builder": "target:b-builder",
        "e": "target:e",
        "p": "target:p",
        "runtime-base": "target:runtime-base"
      },
      "dockerfile": "B/Dockerfile.deploy.Dockerfile",
      "args": {
        "PROJ": "B"
      },
      "platforms": [
        "linux/amd64"
      ]
    },
    "b-builder": {
      "context": ".",
      "contexts": {
        "build-base": "target:build-base"
      },
      "dockerfile": "Dockerfile.builder.Dockerfile",
      "args": {
        "PROJ": "B"
      },
      "platforms": [
        "linux/amd64"
      ]
    },
    "build-base": {
      "context": ".",
      "dockerfile": "Dockerfile.build-base.Dockerfile",
      "platforms": [
        "linux/amd64"
      ]
    },
    "c": {
      "context": ".",
      "contexts": {
        "builder": "target:c-builder",
        "runtime-base": "target:runtime-base"
      },
      "dockerfile": "Dockerfile.all.Dockerfile",
      "args": {
        "PROJ": "X.C"
      },
      "platforms": [
        "linux/amd64"
      ]
    },
    "c-builder": {
      "context": ".",
      "contexts": {
        "build-base": "target:build-base"
      },
      "dockerfile": "Dockerfile.builder.Dockerfile",
      "args": {
        "PROJ": "X.C"
      },
      "platforms": [
        "linux/amd64"
      ]
    },
    "d": {
      "context": ".",
      "contexts": {
        "builder": "target:d-builder",
        "runtime-base": "target:runtime-base"
      },
      "dockerfile": "Dockerfile.all.Dockerfile",
      "args": {
        "PROJ": "X.D"
      },
      "platforms": [
        "linux/amd64"
      ]
    },
    "d-builder": {
      "context": ".",
      "contexts": {
        "build-base": "target:build-base"
      },
      "dockerfile": "Dockerfile.builder.Dockerfile",
      "args": {
        "PROJ": "X.D"
      },
      "platforms": [
        "linux/amd64"
      ]
    },
    "e": {
      "context": "Y/source",
      "dockerfile": "Dockerfile"
    },
    "p": {
      "context": "Z/Z",
      "dockerfile": "Dockerfile"
    },
    "runtime-base": {
      "context": ".",
      "dockerfile": "Dockerfile.runtime-base.Dockerfile",
      "platforms": [
        "linux/amd64"
      ]
    },
    "test": {
      "context": ".",
      "contexts": {
        "builder": "target:test-builder",
        "test-base": "target:test-base"
      },
      "dockerfile": "Dockerfile.all-test.Dockerfile",
      "platforms": [
        "linux/amd64"
      ],
      "output": [
        "type=local,dest=./test-results-output"
      ]
    },
    "test-base": {
      "context": ".",
      "dockerfile": "Dockerfile.test-base.Dockerfile",
      "platforms": [
        "linux/amd64"
      ]
    },
    "test-builder": {
      "context": ".",
      "contexts": {
        "build-base": "target:build-base"
      },
      "dockerfile": "Dockerfile.builder.Dockerfile",
      "args": {
        "PROJ": "X.Test"
      },
      "platforms": [
        "linux/amd64"
      ]
    }
  }
}

Can you please double-check that you're giving the right workflow and please post the full workflow, not just the "Build" step. Better would be to have a link to your GitHub repo so I can check myself the logs. Thanks.

@crazy-max
Copy link
Member

docker/actions-toolkit#276 should fix the provenance issue.

@tonistiigi
Copy link
Member

I think the issue is handling of --push (or maybe same logic is reached in some other way) . In a bake file that defines output = ["type=local,dest=foo"] and then adds --push I can see behavior change where it now changes it to type=registry .

The question is if this is logically correct, especially considering that we now support multiple exporters. By definition --push means setting output for all targets to registry so the new behavior looks correct. Although, if the old behavior differed, the user could thought that it meant "add push output only if one is not set in the bake file". But if we now also support multiple exporters then that doesn't really make sense anymore as well.

@davhdavh can you confirm that --push was set in your output or do we still need to look for another codepath.

@tonistiigi tonistiigi changed the title 0.13 broke build: tag is needed when pushing to registry [0.13] bake: changed behavior in output configuration Mar 8, 2024
@davhdavh
Copy link
Author

davhdavh commented Mar 9, 2024

It really must have been friday yesterday. Yes, forgot to add the last 2 lines of the github job, where push and pull are true.
I suggest you make a --output-force / --output-overwrite or --output-append or--output-whenmissing to control the behaviour of how the command line output should be interpreted. And leave the default as whenmissing as previously.

@crazy-max
Copy link
Member

It really must have been friday yesterday. Yes, forgot to add the last 2 lines of the github job, where push and pull are true. I suggest you make a --output-force / --output-overwrite or --output-append or--output-whenmissing to control the behaviour of how the command line output should be interpreted. And leave the default as whenmissing as previously.

Can you post the full workflow?

Is your GitHub repo public so we can look at it?

Thanks

@tonistiigi
Copy link
Member

@crazy-max This can be reproduced with this simple case.

docker-bake.hcl

target "foo" {
  output = [ "type=local,dest=out" ]
}

target "bar" {
}

v0.13

» docker buildx bake --push --print foo bar                                                                                                                                                           
{
  "group": {
    "default": {
      "targets": [
        "foo",
        "bar"
      ]
    }
  },
  "target": {
    "bar": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "output": [
        "type=registry"
      ]
    },
    "foo": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "output": [
        "type=registry"
      ]
    }
  }
}

v0.12

docker-buildx  bake --push --print foo bar
{
  "group": {
    "default": {
      "targets": [
        "foo",
        "bar"
      ]
    }
  },
  "target": {
    "bar": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "output": [
        "type=image,push=true"
      ]
    },
    "foo": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "output": [
        "type=local,dest=out"
      ]
    }
  }
}

@tonistiigi
Copy link
Member

@crazy-max Note that in v0.12 push=true is also set if output is type=image. So it is empty or image output without push is turned into image output with push. I guess --load is same, but we need to double check.

@crazy-max crazy-max added this to the v0.13.1 milestone Mar 10, 2024
@crazy-max crazy-max added kind/bug Something isn't working and removed status/needs-more-info labels Mar 10, 2024
@crazy-max
Copy link
Member

crazy-max commented Mar 10, 2024

Yes same with --load but gives the same in v0.12 and v0.13:

v0.13

$ docker buildx bake --load --print foo bar
#1 [internal] load local bake definitions
#1 reading docker-bake.hcl 72B / 72B done
#1 DONE 0.0s
{
  "group": {
    "default": {
      "targets": [
        "foo",
        "bar"
      ]
    }
  },
  "target": {
    "bar": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "output": [
        "type=docker"
      ]
    },
    "foo": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "output": [
        "type=docker"
      ]
    }
  }
}

v0.12

$ docker buildx bake --load --print foo bar
#1 [internal] load local bake definitions
#1 reading docker-bake.hcl 72B / 72B done
#1 DONE 0.0s
{
  "group": {
    "default": {
      "targets": [
        "foo",
        "bar"
      ]
    }
  },
  "target": {
    "bar": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "output": [
        "type=docker"
      ]
    },
    "foo": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "output": [
        "type=docker"
      ]
    }
  }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/bake kind/bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants