[spike] `Image`: Implement a selection of `extraOptions` #424

guineveresaenger · 2022-11-28T19:28:03Z

Hello!

Vote on this issue by adding a 👍 reaction
If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

In v3.x.x, we allow users to set extra options which get passed to the Docker provider as command line flags.

Using the client, these all have to be implemented separately.

Determine which options users are currently relying on
Add logic to capture and parse extraOptions
- Add new fields to the schema rather than a grab bag list
- Deprecate the extraOptions field
Pass on options to the client

The text was updated successfully, but these errors were encountered:

LouisAmon · 2023-03-14T17:29:50Z

As mentionned here, I do believe that implementing CacheTo would be immensely helpful to people that want to leverage the GitHub Actions Cache

https://docs.docker.com/build/cache/backends/gha/

uatach · 2023-03-15T21:53:26Z

I think the --network flag would be great.

In GCP Cloud Build we need access to the cloudbuild network to fetch credentials, there's currently an open issue about it in the Google IssueTracker and some discussions of workarounds for BuildKit in docker/buildx#175

AaronFriel · 2023-03-16T21:20:07Z

In #549 a user requested the build use multiple tags and push each of them.

AaronFriel · 2023-03-16T21:20:27Z

In #539 a user requested the --secret option.

LouisAmon · 2023-03-30T11:11:07Z

The --ssh flag is currently missing and preventing us from migrating into pulumi-docker 4.x

We use it to clone a GitHub repo inside our Dockerfile (the SSH key is the main means of authentication with GH)

JohnMcGue · 2023-05-25T15:36:17Z

bump on this. our current image build:

const uploaderImage = registryInfo.apply((registryInfo) => new docker.Image(uploaderImageName, {
    build: {
        context: "../uploader",
        // needed for ssh install of our private repo
        extraOptions: [
            '--ssh', `default=/home/runner/.ssh/id_ed25519`,
        ],
        env: {
            DOCKER_BUILDKIT: "1",
        },
    },
    imageName: pulumi.interpolate`${ecrRepo.repositoryUrl}:${imageName}`,
    registry: registryInfo,
}));

we are also using the SSH key to authenticate with a private GitHub repository. this is blocking us from migrating to 4.x.

yoanisgil · 2023-06-01T20:43:31Z

Not having the ability to pass build secrets is really a major drawback for us for moving with the new Docker provider.

mpuhacz · 2023-07-06T13:35:42Z

Same problem as @JohnMcGue. This is a blocker for migration.

guineveresaenger · 2023-08-25T21:31:34Z

@LouisAmon @JohnMcGue @mpuhacz - I'm not sure if this is useful for your particular situations, but you should be able to use sshOpts with an explicit provider: https://www.pulumi.com/registry/packages/docker/api-docs/provider/#sshopts_nodejs.

This would be a breaking change, but this argument is already not working anymore since it was removed from the underlying Docker provider, see pulumi/pulumi-docker#424. Part of #1148

This would be a breaking change, but this property is already broken since it was removed from the underlying Docker provider, see pulumi/pulumi-docker#424. Part of #1148

rtrindvg · 2024-03-28T18:59:24Z

Is there any option for the quiet parameter? The verbosity on our processes are generating some collateral effects we could avoid otherwise.

AaronFriel · 2024-03-28T20:23:19Z

@rtrindvg Are you using Docker v4.x or v3.x?

rtrindvg · 2024-04-01T15:10:45Z

@rtrindvg Are you using Docker v4.x or v3.x?

I was using v3.x before, using the quiet parameter on the extraOptions. Now we transitioned to v4.x, because we had an issue of generating a new docker image on every preview, which was slowing down the process, but I see no corresponding way of generating the image without a very large output in some complex building process.

AaronFriel · 2024-04-01T23:58:04Z

With v4, you should no longer see a "very large output", do you have an example you can share of a Dockerfile & Pulumi program that produces said output?

rtrindvg · 2024-04-23T18:51:52Z

I could reproduce the issue using a simple repository, and also how there are different behavior when using CLI versus the pulumi/actions github CI module.

Sample index.ts file:

import * as pulumi from "@pulumi/pulumi";
import * as docker from "@pulumi/docker";

const demoImage = new docker.Image("demo-image", {
    build: {
        context: ".",
        dockerfile: "Dockerfile",
        platform: "linux/amd64",
    },
    imageName: "local/demo-verbose-dockerfile:latest",
    skipPush: true,
});
export const imageName = demoImage.imageName;

Sample Dockerfile:

FROM alpine:latest
RUN for i in $(seq 1 1000); do echo "This is a verbose output: line $i"; done

Pulumi CLI preview (no issue, even with diff, since the preview does not build the image):

❯ pulumi preview --diff
Previewing update (temp2-stage):
+ pulumi:pulumi:Stack: (create)
    [urn=urn:pulumi:temp2-stage::pulumi::pulumi:pulumi:Stack::pulumi-temp2-stage]
    + docker:index/image:Image: (create)
        [urn=urn:pulumi:temp2-stage::pulumi::docker:index/image:Image::demo-image]
        [provider=urn:pulumi:temp2-stage::pulumi::pulumi:providers:docker::default_4_5_3::04da6b54-80e4-46f7-96ec-b56ff0331ba9]
        build         : {
            context      : "."
            contextDigest: "14b6a600929aa0bb7399328800e0c8b47aa9c93945203083302b9e9383967337"
            dockerfile   : "Dockerfile"
            platform     : "linux/amd64"
        }
        buildOnPreview: false
        imageName     : "demoVerboseDockerfile:latest"
        skipPush      : true
    --outputs:--
    imageName: "demoVerboseDockerfile:latest"
Resources:
    + 2 to create

A regular Pulumi CLI up won't have the problem, since the build messages are replaced on the same line on the screen:

❯ pulumi up -f
Updating (temp-stage):
     Type                   Name               Status           
     pulumi:pulumi:Stack    pulumi-temp-stage                   
 +   └─ docker:index:Image  demo-image         created (2s)     

Outputs:
  + imageName: "local/demo-verbose-dockerfile:latest"

Resources:
    + 1 created
    1 unchanged

Duration: 6s

... but when I use the diff parameter we have a partial problem, not all expected output is here (since my RUN lines are not shown on the output) but there is still a lot of lines here. In a real Dockerfile, with all the download steps and progress, this is already a lot of output:

❯ pulumi up --diff -f
Updating (temp-stage):
  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:temp-stage::pulumi::pulumi:pulumi:Stack::pulumi-temp-stage]
    ~ docker:index/image:Image: (update)
        [id=local/demo-verbose-dockerfile:latest]
        [urn=urn:pulumi:temp-stage::pulumi::docker:index/image:Image::demo-image]
        [provider=urn:pulumi:temp-stage::pulumi::pulumi:providers:docker::default_4_5_3::e3b8ed63-8c00-45e7-ad5f-9f37857620b2]
      + build: {
          + context      : "."
          + contextDigest: "06df9819d5e98abc5508a37291c43e2571b15504a8f865a972b08dcefd43d564"
          + dockerfile   : "Dockerfile"
          + platform     : "linux/amd64"
        }
Starting Docker build
digest: sha256:bc7d86e8ed6e2ea2c3e5c58427a68b17ec9a344cff43e93bba78100f7c452aa8
[internal] load remote build context
digest: sha256:bc7d86e8ed6e2ea2c3e5c58427a68b17ec9a344cff43e93bba78100f7c452aa8
[internal] load remote build context
digest: sha256:bc7d86e8ed6e2ea2c3e5c58427a68b17ec9a344cff43e93bba78100f7c452aa8
[internal] load remote build context
digest: sha256:bc7d86e8ed6e2ea2c3e5c58427a68b17ec9a344cff43e93bba78100f7c452aa8
[internal] load remote build context
digest: sha256:b3997bdf0efd40151cdfb63a5b1c84cd89f5cfca69d79cf152696cdc63c2074c
copy /context /
digest: sha256:b3997bdf0efd40151cdfb63a5b1c84cd89f5cfca69d79cf152696cdc63c2074c
copy /context /
digest: sha256:b3997bdf0efd40151cdfb63a5b1c84cd89f5cfca69d79cf152696cdc63c2074c
copy /context /
digest: sha256:038fafa2e280d24ac22549b7cef4c77fc469b484de8826b26b10bc92efa2f54d
[internal] load metadata for docker.io/library/alpine:latest
digest: sha256:038fafa2e280d24ac22549b7cef4c77fc469b484de8826b26b10bc92efa2f54d
[internal] load metadata for docker.io/library/alpine:latest
digest: sha256:86959bee9189c6af7dc145e532ec0ba6bab7a0c67d9675b42acbc89fe1df1bf7
[2/2] RUN for i in $(seq 1 1000); do echo "This is a verbose output: line $i"; done
digest: sha256:faa2314417184f0f81e21fe46ef9c04da17b8fc0d629ac5f017dd02a596697bb
[1/2] FROM docker.io/library/alpine:latest@sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
digest: sha256:faa2314417184f0f81e21fe46ef9c04da17b8fc0d629ac5f017dd02a596697bb
[1/2] FROM docker.io/library/alpine:latest@sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
digest: sha256:86959bee9189c6af7dc145e532ec0ba6bab7a0c67d9675b42acbc89fe1df1bf7
[2/2] RUN for i in $(seq 1 1000); do echo "This is a verbose output: line $i"; done
digest: sha256:42f35dcbdcf8c4f6184d5f8823401b03fe416f8864cd88b7b145ab9427a4669c
exporting to image
exporting layers
writing image sha256:bbbe99bfe0ec6aff64100950a62c57f864739459142a5e404c9ce2255741368f
writing image sha256:bbbe99bfe0ec6aff64100950a62c57f864739459142a5e404c9ce2255741368f
naming to docker.io/local/demo-verbose-dockerfile:latest
naming to docker.io/local/demo-verbose-dockerfile:latest
digest: sha256:42f35dcbdcf8c4f6184d5f8823401b03fe416f8864cd88b7b145ab9427a4669c
exporting to image
Built image with ID sha256:bbbe99bfe0ec6aff64100950a62c57f864739459142a5e404c9ce2255741368f
Built image with local id "sha256:bbbe99bfe0ec6aff64100950a62c57f864739459142a5e404c9ce2255741368f", polling image store for image
Image built successfully, local id "sha256:bbbe99bfe0ec6aff64100950a62c57f864739459142a5e404c9ce2255741368f"
        --outputs:--
      ~ repoDigest    : "sha256:ad161c6c986f7f462700ec43dca223d5e23d35685173322f42ef87194fe8754e" => "sha256:bbbe99bfe0ec6aff64100950a62c57f864739459142a5e404c9ce2255741368f"
    --outputs:--
    imageName: "local/demo-verbose-dockerfile:latest"
Resources:
    ~ 1 updated
    1 unchanged

Duration: 6s

Note the verbose output from the RUN command in the docker file somehow is not here, this is the same execution on the CI, using pulumi/actions (with diff: true):

pulumi up on temp-stage
  Updating (temp-stage):
  
    pulumi:pulumi:Stack: (same)
      [urn=urn:pulumi:temp-stage::pulumi::pulumi:pulumi:Stack::pulumi-temp-stage]
  
      ~ docker:index/image:Image: (update)
          [id=local/demo-verbose-dockerfile:latest]
          [urn=urn:pulumi:temp-stage::pulumi::docker:index/image:Image::demo-image]
          [provider=urn:pulumi:temp-stage::pulumi::pulumi:providers:docker::default_4_5_3::e3b8ed63-8c00-45e7-ad5f-9f37857620b2]
        + build: {
            + context      : "."
            + contextDigest: "c62efd5d3930b3d15060cac4257a8935eec0acc895e08e0de3e5dde8ff8da669"
            + dockerfile   : "Dockerfile"
            + platform     : "linux/amd64"
          }
  
  Starting Docker build
  
  digest: sha256:3c349566976773845862abb6848c2fbd7542538581cca0d3a485f73b57c47bc5
  [internal] load remote build context
  
  digest: sha256:3c349566976773845862abb6848c2fbd7542538581cca0d3a485f73b57c47bc5
  [internal] load remote build context
  
  digest: sha256:3c349566976773845862abb6848c2fbd7542538581cca0d3a485f73b57c47bc5
  [internal] load remote build context
  
  digest: sha256:3c349566976773845862abb6848c2fbd7542538581cca0d3a485f73b57c47bc5
  [internal] load remote build context
  
  digest: sha256:716142e0bc6cb7d8c05f15a25af9318ad32cf96431bf08f931556292701341cb
  copy /context /
  
  digest: sha256:716142e0bc6cb7d8c05f15a25af9318ad32cf96431bf08f931556292701341cb
  copy /context /
  
  digest: sha256:716142e0bc6cb7d8c05f15a25af9318ad32cf96431bf08f931556292701341cb
  copy /context /
  
  digest: sha256:d4fb25f5b5c00defc20ce26f2efc4e288de8834ed5aa59dff877b495ba88fda6
  [internal] load metadata for docker.io/library/alpine:latest
  
  digest: sha256:d4fb25f5b5c00defc20ce26f2efc4e288de8834ed5aa59dff877b495ba88fda6
  [internal] load metadata for docker.io/library/alpine:latest
  
  digest: sha256:86959bee9189c6af7dc145e532ec0ba6bab7a0c67d9675b42acbc89fe1df1bf7
  [2/2] RUN for i in $(seq 1 1000); do echo "This is a verbose output: line $i"; done
  digest: sha256:faa2314417184f0f81e21fe46ef9c04da17b8fc0d629ac5f017dd02a596697bb
  [1/2] FROM docker.io/library/alpine:latest@sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
  
  digest: sha256:faa2314417184f0f81e21fe46ef9c04da17b8fc0d629ac5f017dd02a596697bb
  [1/2] FROM docker.io/library/alpine:latest@sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
  digest: sha256:faa2314417184f0f81e21fe46ef9c04da17b8fc0d629ac5f017dd02a596697bb
  [1/2] FROM docker.io/library/alpine:latest@sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
  
  resolve docker.io/library/alpine:latest@sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
  
  resolve docker.io/library/alpine:latest@sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
  
  digest: sha256:faa2314417184f0f81e21fe46ef9c04da17b8fc0d629ac5f017dd02a596697bb
  [1/2] FROM docker.io/library/alpine:latest@sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
  
  digest: sha256:faa2314417184f0f81e21fe46ef9c04da17b8fc0d629ac5f017dd02a596697bb
  [1/2] FROM docker.io/library/alpine:latest@sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
  
  sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
  sha256:6457d53fb065d6f250e1504b9bc42d5b6c65941d57532c072d929dd0628977d0
  sha256:05455a08881ea9cf0e752bc48e61bbd71a34c029bb13df01e40e3e70e0d007bd
  
  digest: sha256:faa2314417184f0f81e21fe46ef9c04da17b8fc0d629ac5f017dd02a596697bb
  [1/2] FROM docker.io/library/alpine:latest@sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
  
  digest: sha256:86959bee9189c6af7dc145e532ec0ba6bab7a0c67d9675b42acbc89fe1df1bf7
  [2/2] RUN for i in $(seq 1 1000); do echo "This is a verbose output: line $i"; done
  
  This is a verbose output: line 1
  
  This is a verbose output: line 2
  This is a verbose output: line 3
  This is a verbose output: line 4
... <cut for brevity> ...
  This is a verbose output: line 998
  This is a verbose output: line 999
  This is a verbose output: line 1000
  
  digest: sha256:86959bee9189c6af7dc145e532ec0ba6bab7a0c67d9675b42acbc89fe1df1bf7
  [2/2] RUN for i in $(seq 1 1000); do echo "This is a verbose output: line $i"; done
  
  digest: sha256:86f50521e51c3b2bab702287bad76a1a78e381a965e20c8b641d08a6cc2d18b8
  exporting to image
  exporting layers
  
  exporting layers
  
  writing image sha256:ad161c6c986f7f462700ec43dca223d5e23d35685173322f42ef87194fe8754e
  writing image sha256:ad161c6c986f7f462700ec43dca223d5e23d35685173322f42ef87194fe8754e
  naming to docker.io/local/demo-verbose-dockerfile:latest
  
  digest: sha256:86f50521e51c3b2bab702287bad76a1a78e381a965e20c8b641d08a6cc2d18b8
  exporting to image
  naming to docker.io/local/demo-verbose-dockerfile:latest
  
  Built image with ID sha256:ad161c6c986f7f462700ec43dca223d5e23d35685173322f42ef87194fe8754e
  
  Built image with local id "sha256:ad161c6c986f7f462700ec43dca223d5e23d35685173322f42ef87194fe8754e", polling image store for image
  
  Image built successfully, local id "sha256:ad161c6c986f7f462700ec43dca223d5e23d35685173322f42ef87194fe8754e"
  
          --outputs:--
        ~ repoDigest    : "sha256:e385eeb5c8043e094168b13a3b6feac765d19e4a57f1f84ac34220de18c4b255" => "sha256:ad161c6c986f7f462700ec43dca223d5e23d35685173322f42ef87194fe8754e"
  
      --outputs:--
      imageName: "local/demo-verbose-dockerfile:latest"
  
  Resources:
      ~ 1 updated
      1 unchanged
  
  Duration: 8s

Same execution with diff: false, still very verbose:

pulumi up on temp-stage
  Updating (temp-stage):
  
  
  
      pulumi:pulumi:Stack pulumi-temp-stage running 
  
   ~  docker:index:Image demo-image updating (0s) [diff: ~build]
  
   ~  docker:index:Image demo-image updating (0s) [diff: ~build]; Starting Docker build
  
   ~  docker:index:Image demo-image updating (0s) [diff: ~build]; digest: sha256:426e7f1b4791ef9aaddabf8c62b15184c6945da080b8207b7c2a84f791c5b273
  
   ~  docker:index:Image demo-image updating (0s) [diff: ~build]; digest: sha256:426e7f1b4791ef9aaddabf8c62b15184c6945da080b8207b7c2a84f791c5b273
  
   ~  docker:index:Image demo-image updating (0s) [diff: ~build]; digest: sha256:426e7f1b4791ef9aaddabf8c62b15184c6945da080b8207b7c2a84f791c5b273
  
   ~  docker:index:Image demo-image updating (0s) [diff: ~build]; digest: sha256:426e7f1b4791ef9aaddabf8c62b15184c6945da080b8207b7c2a84f791c5b273
  
   ~  docker:index:Image demo-image updating (0s) [diff: ~build]; digest: sha256:3cdec11ab05d96670ce11e206b0f7b0ec34eb947d74feb337161ae102a6274fe
  
   ~  docker:index:Image demo-image updating (0s) [diff: ~build]; digest: sha256:3cdec11ab05d96670ce11e206b0f7b0ec34eb947d74feb337161ae102a6274fe
  
   ~  docker:index:Image demo-image updating (1s) [diff: ~build]; digest: sha256:3cdec11ab05d96670ce11e206b0f7b0ec34eb947d74feb337161ae102a6274fe
  
   ~  docker:index:Image demo-image updating (1s) [diff: ~build]; digest: sha256:d4fb25f5b5c00defc20ce26f2efc4e288de8834ed5aa59dff877b495ba88fda6
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; digest: sha256:d4fb25f5b5c00defc20ce26f2efc4e288de8834ed5aa59dff877b495ba88fda6
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; digest: sha256:86959bee9189c6af7dc145e532ec0ba6bab7a0c67d9675b42acbc89fe1df1bf7
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; digest: sha256:faa2314417184f0f81e21fe46ef9c04da17b8fc0d629ac5f017dd02a596697bb
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; resolve docker.io/library/alpine:latest@sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; digest: sha256:faa2314417184f0f81e21fe46ef9c04da17b8fc0d629ac5f017dd02a596697bb
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; digest: sha256:faa2314417184f0f81e21fe46ef9c04da17b8fc0d629ac5f017dd02a596697bb
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; digest: sha256:faa2314417184f0f81e21fe46ef9c04da17b8fc0d629ac5f017dd02a596697bb
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; digest: sha256:86959bee9189c6af7dc145e532ec0ba6bab7a0c67d9675b42acbc89fe1df1bf7
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 1
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 3
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 27
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 34
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 38
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 42
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 48
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 51
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 53
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 58
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 66
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 70
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 75
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 80
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 84
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 89
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 92
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 94
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 101
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 137
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 178
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 187
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 233
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 242
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 284
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 323
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 334
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 374
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 412
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 422
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 459
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 465
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 499
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 542
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 552
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 593
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 622
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 631
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 673
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 691
  
   ~  docker:index:Image demo-image updating (2s) [diff: ~build]; This is a verbose output: line 738
  
   ~  docker:index:Image demo-image updating (3s) [diff: ~build]; digest: sha256:86959bee9189c6af7dc145e532ec0ba6bab7a0c67d9675b42acbc89fe1df1bf7
  
   ~  docker:index:Image demo-image updating (3s) [diff: ~build]; digest: sha256:e7aebe2d8fbfef621b9a4901ad9b5490eb67b477264bd48a0fac928efe3aed93
  
   ~  docker:index:Image demo-image updating (3s) [diff: ~build]; exporting layers
  
   ~  docker:index:Image demo-image updating (3s) [diff: ~build]; writing image sha256:d1e3baaa05cb378327d95d861e5daa798d6b20f0adc89edf09916138161282b4
  
   ~  docker:index:Image demo-image updating (3s) [diff: ~build]; writing image sha256:d1e3baaa05cb378327d95d861e5daa798d6b20f0adc89edf09916138161282b4
  
   ~  docker:index:Image demo-image updating (3s) [diff: ~build]; digest: sha256:e7aebe2d8fbfef621b9a4901ad9b5490eb67b477264bd48a0fac928efe3aed93
  
   ~  docker:index:Image demo-image updating (3s) [diff: ~build]; Built image with ID sha256:d1e3baaa05cb378327d95d861e5daa798d6b20f0adc89edf09916138161282b4
  
   ~  docker:index:Image demo-image updating (3s) [diff: ~build]; Built image with local id "sha256:d1e3baaa05cb378327d95d861e5daa798d6b20f0adc89edf09916138161282b4", polling image store for image
  
   ~  docker:index:Image demo-image updating (3s) [diff: ~build]; Image built successfully, local id "sha256:d1e3baaa05cb378327d95d861e5daa798d6b20f0adc89edf09916138161282b4"
  
   ~  docker:index:Image demo-image updated (3s) [diff: ~build]; Image built successfully, local id "sha256:d1e3baaa05cb378327d95d861e5daa798d6b20f0adc89edf09916138161282b4"
  
      pulumi:pulumi:Stack pulumi-temp-stage  
  Outputs:
      imageName: "local/demo-verbose-dockerfile:latest"
  
  Resources:
      ~ 1 updated
      1 unchanged

I am already using suppress-progress on pulumi/actions (excerpt from my workflow file):

... cut for brevity ...
    - name: Pulumi Command
      id: pulumi
      uses: pulumi/actions@v5.2.2
      with:
        command: up
        stack-name: ${{ inputs.PULUMI_STACK_NAME }}
        work-dir: ${{ inputs.WORKING_DIR }}
        cloud-url: azblob://pulumi-state?storage_account=<redacted>
        comment-on-pr: true
        edit-pr-comment: false
        diff: ${{ inputs.DIFF }}
        upsert: true
        secrets-provider: azurekeyvault://<redacted>
        suppress-progress: true
      env:
        ARM_USE_OIDC: true
        ARM_CLIENT_ID: ${{ inputs.AZURE_CLIENT_ID }}
        ARM_TENANT_ID: ${{ inputs.AZURE_TENANT_ID }}
        ARM_SUBSCRIPTION_ID: ${{ inputs.AZURE_SUBSCRIPTION_ID }}
        AZURE_KEYVAULT_AUTH_VIA_CLI: true

In summary, I need diff for audit purposes, but even without it, since docker v4 I have no way of achieving this without a mess. You can imagine how large Dockerfiles, with unit tests embedded on them, make a lot of lines of output, which in turn makes them almost unusable on the PR comments, since the relevant information is buried on them.

Having a more uniform behavior for diff true/false on both CLI and pulumi/actions is a partial win, but I cannot see how to achieve what I need without an explicit way to control the docker build output in some level. Even quiet is not perfect, since we can lose error messages from the docker build process, but it's much better than what we have right now.

Hope it reproducible on your side now and somehow this parameter (or another equivalent one) can be provided eventually.

AaronFriel · 2024-04-24T01:07:33Z

I could reproduce the issue using a simple repository, and also how there are different behavior when using CLI versus the pulumi/actions github CI module.

Definitely, this seems like a feature request for the Pulumi CLI, as it looks like different types of events are logged with --diff depending on whether it's a pty or not. Please open an issue here on http://github.com/pulumi/pulumi!

If you're able to look into using the Pulumi Cloud backend, you'll see that by integrating with the Pulumi engine, we can toggle between toggle between verbose, diff and diagnostic outputs. Here's what that looks like:

Where selecting "Diff" or "Diagnostic" will provide more verbose output, which I think matches what you're seeing.

rtrindvg · 2024-04-24T13:06:33Z

Thanks, will open the issue there. But even making this consistent on pulumi CLI, I still believe the need to have the quiet option exposed continues, since the changes on the CLI won't help me at all, since my true issue is on the CI. I will also open an issue on pulumi/actions, to make sure the diff=false scenario is correct (I believe it should not print the progress from docker build), but the need to have the quiet parameter will also persist (since I need diff=true).

Unfortunately Pulumi Cloud backend is outside of our budget for now, but it would be perfect, for sure.

Thanks for the feedback!

blampe · 2024-04-25T18:21:59Z

Hi folks, we just released a new Docker Build provider focused exclusively on building images with buildx/BuildKit. It provides --ssh, --cache-to, multiple tags, and quieter logs out of the box. We recommend you use it for building images going forward. Please take a look and give us feedback!

The repository is here; our blog announcement is here; and finally API docs are here along with examples of how to migrate your existing Image resources to dockerbuild.Image.

We won't be able to add this functionality in this provider for a number of reasons, so I'm closing this as wont-fix.

guineveresaenger added kind/enhancement Improvements or new features 4.x.x labels Nov 28, 2022

This was referenced Jan 27, 2023

Clean up env and extraOptions types #469

Closed

Image: Implement crossplatform builds #470

Closed

AaronFriel mentioned this issue Mar 16, 2023

Version 4.x: Additional tags are not pushed remotely #549

Open

willgiang147 mentioned this issue Sep 25, 2023

Add support for --ssh in Pulumi Docker V4.x.x #761

Closed

mikhailshilkov mentioned this issue Nov 4, 2023

extraOptions is ignored when using awsx.ecr.Image pulumi/pulumi-awsx#1148

Closed

thomas11 mentioned this issue Nov 9, 2023

Remove awsx.Ecr.ImageArgs.extraOptions pulumi/pulumi-awsx#1150

Merged

uatach mentioned this issue Nov 9, 2023

Add support for --network in Pulumi Docker v4 using BuilderV1 #837

Closed

mjeffryes mentioned this issue Feb 13, 2024

[Epic] Docker-build Provider #963

Closed

rtrindvg mentioned this issue Apr 24, 2024

In CLI different types of events are logged with --diff depending on whether it's a pty or not pulumi/pulumi#16046

Open

blampe added the resolution/wont-fix This issue won't be fixed label Apr 25, 2024

blampe closed this as completed Apr 25, 2024

mjeffryes assigned blampe Apr 26, 2024

justinvp mentioned this issue May 6, 2024

When "diff: false", the progress from pulumi/docker@v4 build command is still shown pulumi/actions#1157

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[spike] `Image`: Implement a selection of `extraOptions` #424

[spike] `Image`: Implement a selection of `extraOptions` #424

guineveresaenger commented Nov 28, 2022 •

edited

Loading

LouisAmon commented Mar 14, 2023

uatach commented Mar 15, 2023

AaronFriel commented Mar 16, 2023

AaronFriel commented Mar 16, 2023

LouisAmon commented Mar 30, 2023 •

edited

Loading

JohnMcGue commented May 25, 2023 •

edited

Loading

yoanisgil commented Jun 1, 2023 •

edited

Loading

mpuhacz commented Jul 6, 2023

guineveresaenger commented Aug 25, 2023

rtrindvg commented Mar 28, 2024

AaronFriel commented Mar 28, 2024

rtrindvg commented Apr 1, 2024

AaronFriel commented Apr 1, 2024

rtrindvg commented Apr 23, 2024

AaronFriel commented Apr 24, 2024

rtrindvg commented Apr 24, 2024 •

edited

Loading

blampe commented Apr 25, 2024

[spike] Image: Implement a selection of extraOptions #424

[spike] Image: Implement a selection of extraOptions #424

Comments

guineveresaenger commented Nov 28, 2022 • edited Loading

Hello!

Issue details

LouisAmon commented Mar 14, 2023

uatach commented Mar 15, 2023

AaronFriel commented Mar 16, 2023

AaronFriel commented Mar 16, 2023

LouisAmon commented Mar 30, 2023 • edited Loading

JohnMcGue commented May 25, 2023 • edited Loading

yoanisgil commented Jun 1, 2023 • edited Loading

mpuhacz commented Jul 6, 2023

guineveresaenger commented Aug 25, 2023

rtrindvg commented Mar 28, 2024

AaronFriel commented Mar 28, 2024

rtrindvg commented Apr 1, 2024

AaronFriel commented Apr 1, 2024

rtrindvg commented Apr 23, 2024

AaronFriel commented Apr 24, 2024

rtrindvg commented Apr 24, 2024 • edited Loading

blampe commented Apr 25, 2024

[spike] `Image`: Implement a selection of `extraOptions` #424

[spike] `Image`: Implement a selection of `extraOptions` #424

guineveresaenger commented Nov 28, 2022 •

edited

Loading

LouisAmon commented Mar 30, 2023 •

edited

Loading

JohnMcGue commented May 25, 2023 •

edited

Loading

yoanisgil commented Jun 1, 2023 •

edited

Loading

rtrindvg commented Apr 24, 2024 •

edited

Loading