Allow users to set a specific ContainerImageFormat to force OCI support #46011

baronfel · 2025-01-15T21:18:09Z

This introduces a new MSBuild property, ContainerImageFormat, which can be one of OCI or Docker. If present, this will force single image manifests and configuration to use the format specified. If unset, the formats of the base image will be used.

I was able to test this locally and verify all of the expected media types on a locally-running registry:2 instance

(dogfood) PS E:\Code\sdk-container-demo\src\sdk-container-demo> docker run -d -p 5000:5000 --name registry registry:2
Unable to find image 'registry:2' locally
2: Pulling from library/registry
9959184a302f: Download complete
c8f4e00e7d3c: Download complete
665375f37302: Download complete
b6afea20d55c: Download complete
f54a5150a760: Download complete
Digest: sha256:319881be2ee9e345d5837d15842a04268de6a139e23be42654fc7664fc6eaf52
Status: Downloaded newer image for registry:2
acac5772a8c31fcb36d1cd921701354c88f436cb476522f2bf94669c43475419

> dotnet publish -t:PublishContainer -r linux-x64 -v d -bl -p ContainerRegistry=localhost:5000
Restore complete (0.8s)
    Determining projects to restore...
    Restored E:\Code\sdk-container-demo\src\sdk-container-demo\sdk-container-demo.csproj (in 240 ms).
You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy
  sdk-container-demo succeeded (1.0s) → E:\Code\sdk-container-demo\artifacts\publish\sdk-container-demo\release_linux-x64\
  sdk-container-demo succeeded (1.7s)
    Building image 'sdk-container-demo' with tags 'latest,v1.1' on top of base image 'mcr.microsoft.com/dotnet/aspnet:8.0'.
    Uploading layer 'sha256:af302e5c37e9dc1dbe2eadc8f5059d82a914066b541b0d1a6daa91d0cc55057d' to 'localhost:5000'.
    Uploading layer 'sha256:91ab5e0aabf0f167bbdba019f1b60b9548e08fdf67363574c6b82e6092103e57' to 'localhost:5000'.
    Uploading layer 'sha256:1c1e4530721eb878290d29bdf34361bf78cc59c7327025c2fe28864a63f3711e' to 'localhost:5000'.
    Uploading layer 'sha256:1f39ca6dcc3a12c21f7199d627b864351ac61c0db49df3a25de0e15450a02f80' to 'localhost:5000'.
    Uploading layer 'sha256:ea20083aa80192d617a64b64459368605749911195831cf824197dddc2a1961c' to 'localhost:5000'.
    Uploading layer 'sha256:64c242a4f5610cb4ad633229aa50fa34290f74842cf6a32ae6dd31d79f827757' to 'localhost:5000'.
    Uploading layer 'sha256:885ece27c2d30d127979e3a15ac05efef79df2998126483a0b1bd03628749972' to 'localhost:5000'.
    Layer 'sha256:af302e5c37e9dc1dbe2eadc8f5059d82a914066b541b0d1a6daa91d0cc55057d' already exists.
    Layer 'sha256:64c242a4f5610cb4ad633229aa50fa34290f74842cf6a32ae6dd31d79f827757' already exists.
    Layer 'sha256:91ab5e0aabf0f167bbdba019f1b60b9548e08fdf67363574c6b82e6092103e57' already exists.
    Layer 'sha256:1f39ca6dcc3a12c21f7199d627b864351ac61c0db49df3a25de0e15450a02f80' already exists.
    Layer 'sha256:ea20083aa80192d617a64b64459368605749911195831cf824197dddc2a1961c' already exists.
    Layer 'sha256:1c1e4530721eb878290d29bdf34361bf78cc59c7327025c2fe28864a63f3711e' already exists.
    Finished uploading layer 'sha256:885ece27c2d30d127979e3a15ac05efef79df2998126483a0b1bd03628749972' to 'localhost:5000'.
    Uploading config to registry at blob 'sha256:9229a63791908fbdac174ef991638983bcf739aa03c5a083e03df5863b8fdafd',
    Uploaded config to registry.
    Uploading tag 'latest' to 'localhost:5000'.
    Uploaded tag 'latest' to 'localhost:5000'.
    Uploading tag 'v1.1' to 'localhost:5000'.
    Uploaded tag 'v1.1' to 'localhost:5000'.
    Pushed image 'sdk-container-demo:latest, v1.1' to registry 'localhost:5000'.

Build succeeded in 3.7s

(dogfood) PS E:\Code\sdk-container-demo\src\sdk-container-demo> curl localhost:5000/v2/sdk-container-demo/manifests/latest
{"errors":[{"code":"MANIFEST_UNKNOWN","message":"OCI manifest found, but accept header does not support OCI manifests"}]}

(dogfood) PS E:\Code\sdk-container-demo\src\sdk-container-demo> curl --header 'Accept: application/vnd.oci.image.manifest.v1+json' localhost:5000/v2/sdk-container-demo/manifests/latest | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1402  100  1402    0     0  44092      0 --:--:-- --:--:-- --:--:-- 45225
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "config": {
    "mediaType": "application/vnd.oci.image.config.v1+json",
    "size": 4169,
    "digest": "sha256:9229a63791908fbdac174ef991638983bcf739aa03c5a083e03df5863b8fdafd"
  },
  "layers": [
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 28212417,
      "digest": "sha256:af302e5c37e9dc1dbe2eadc8f5059d82a914066b541b0d1a6daa91d0cc55057d"
    },
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 18722950,
      "digest": "sha256:91ab5e0aabf0f167bbdba019f1b60b9548e08fdf67363574c6b82e6092103e57"
    },
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 3278,
      "digest": "sha256:1c1e4530721eb878290d29bdf34361bf78cc59c7327025c2fe28864a63f3711e"
    },
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 32244715,
      "digest": "sha256:1f39ca6dcc3a12c21f7199d627b864351ac61c0db49df3a25de0e15450a02f80"
    },
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 153,
      "digest": "sha256:ea20083aa80192d617a64b64459368605749911195831cf824197dddc2a1961c"
    },
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 11069200,
      "digest": "sha256:64c242a4f5610cb4ad633229aa50fa34290f74842cf6a32ae6dd31d79f827757"
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "size": 9377902,
      "digest": "sha256:885ece27c2d30d127979e3a15ac05efef79df2998126483a0b1bd03628749972"
    }
  ]
}

(dogfood) PS E:\Code\sdk-container-demo\src\sdk-container-demo> curl --header 'Accept: application/vnd.oci.image.config.v1+json' localhost:5000/v2/sdk-container-demo/blobs/sha256:9229a63791908fbdac174ef991638983bcf739aa03c5a083e03df5863b8fdafd | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4169  100  4169    0     0   234k      0 --:--:-- --:--:-- --:--:--  239k
{
  "config": {
    "ExposedPorts": {
      "8080/tcp": {}
    },
    "Labels": {
      "org.opencontainers.image.created": "2025-01-15T21:11:53.7672978Z",
      "org.opencontainers.artifact.created": "2025-01-15T21:11:53.7672978Z",
      "org.opencontainers.artifact.description": "\r\n\t\t\tA project that demonstrates publishing to various container registries using just\r\n\t\t\tthe .NET SDK\r\n\t\t",
      "org.opencontainers.image.description": "\r\n\t\t\tA project that demonstrates publishing to various container registries using just\r\n\t\t\tthe .NET SDK\r\n\t\t",
      "org.opencontainers.image.authors": "Chet Husk",
      "org.opencontainers.image.url": "https://github.com/baronfel/sdk-container-demo",
      "org.opencontainers.image.documentation": "https://github.com/baronfel/sdk-container-demo",
      "org.opencontainers.image.version": "1.0.0",
      "org.opencontainers.image.licenses": "MIT",
      "org.opencontainers.image.title": ".NET SDK 8 Container Demo",
      "org.opencontainers.image.base.name": "mcr.microsoft.com/dotnet/aspnet:8.0",
      "net.dot.runtime.majorminor": "8.0",
      "net.dot.sdk.version": "10.0.100-dev",
      "org.opencontainers.image.source": "https://github.com/baronfel/sdk-container-demo",
      "org.opencontainers.image.revision": "530356ee6891b317b316d1c98199e6fcc50341ef",
      "org.opencontainers.image.base.digest": "sha256:2cd76b1aba9a5ee9013baeed7cb52784cfd8198a8f343f5264daa9db7b73f5de"
    },
    "Env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
      "APP_UID=1654",
      "ASPNETCORE_HTTP_PORTS=8080",
      "DOTNET_RUNNING_IN_CONTAINER=true",
      "DOTNET_VERSION=8.0.12",
      "ASPNET_VERSION=8.0.12"
    ],
    "WorkingDir": "/app/",
    "Entrypoint": [
      "dotnet",
      "/app/sdk-container-demo.dll"
    ],
    "User": "1654"
  },
  "created": "2025-01-15T21:11:55.2438910Z",
  "rootfs": {
    "type": "layers",
    "diff_ids": [
      "sha256:f5fe472da25334617e6e6467c7ebce41e0ae5580e5bd0ecbf0d573bacd560ecb",
      "sha256:b138fdfd1095780b78eb87d172392eaedb62c02443fb29e6955d69894d13667c",
      "sha256:bc198994fabd5cb872694fddb790802e81cfbfc7af240138561ae30308bc93c8",
      "sha256:0fbbfdee9cddbc405d7577d75a38028f9ada2fc968343e0d7eaea6d88d7f1cc4",
      "sha256:9f71735511ce298381e7a7cffbbd340976a328622d0fb5ac01e8c4afd7ba9c1d",
      "sha256:f07c146df8a618e310357dccc538d52ee52bddb1f5f6fff2113e47cb9efbd8cc",
      "sha256:a65e489c8479e8d70167886170579ab6dc0c7b6ba7c0c2fec6dee02ddcbca555"
    ]
  },
  "architecture": "amd64",
  "os": "linux",
  "history": [
    {
      "comment": "debuerreotype 0.15",
      "created": "2025-01-13T00:00:00.0000000Z",
      "created_by": "# debian.sh --arch 'amd64' out/ 'bookworm' '@1736726400'"
    },
    {
      "comment": "buildkit.dockerfile.v0",
      "created": "2025-01-14T19:00:44.0220201Z",
      "created_by": "ENV APP_UID=1654 ASPNETCORE_HTTP_PORTS=8080 DOTNET_RUNNING_IN_CONTAINER=true",
      "empty_layer": true
    },
    {
      "comment": "buildkit.dockerfile.v0",
      "created": "2025-01-14T19:00:44.0220201Z",
      "created_by": "RUN /bin/sh -c apt-get update     && apt-get install -y --no-install-recommends         ca-certificates                 libc6         libgcc-s1         libicu72         libssl3         libstdc++6         tzdata         zlib1g     && rm -rf /var/lib/apt/lists/* # buildkit"
    },
    {
      "comment": "buildkit.dockerfile.v0",
      "created": "2025-01-14T19:00:45.9027407Z",
      "created_by": "RUN /bin/sh -c groupadd         --gid=$APP_UID         app     && useradd -l         --uid=$APP_UID         --gid=$APP_UID         --create-home         app # buildkit"
    },
    {
      "comment": "buildkit.dockerfile.v0",
      "created": "2025-01-14T19:00:52.9595288Z",
      "created_by": "ENV DOTNET_VERSION=8.0.12",
      "empty_layer": true
    },
    {
      "comment": "buildkit.dockerfile.v0",
      "created": "2025-01-14T19:00:52.9595288Z",
      "created_by": "COPY /dotnet /usr/share/dotnet # buildkit"
    },
    {
      "comment": "buildkit.dockerfile.v0",
      "created": "2025-01-14T19:00:54.4897837Z",
      "created_by": "RUN /bin/sh -c ln -s /usr/share/dotnet/dotnet /usr/bin/dotnet # buildkit"
    },
    {
      "comment": "buildkit.dockerfile.v0",
      "created": "2025-01-14T19:01:00.3575580Z",
      "created_by": "ENV ASPNET_VERSION=8.0.12",
      "empty_layer": true
    },
    {
      "comment": "buildkit.dockerfile.v0",
      "created": "2025-01-14T19:01:00.3575580Z",
      "created_by": "COPY /shared/Microsoft.AspNetCore.App /usr/share/dotnet/shared/Microsoft.AspNetCore.App # buildkit"
    },
    {
      "author": ".NET SDK",
      "created": "2025-01-15T21:11:55.2438835Z",
      "created_by": ".NET SDK Container Tooling, version 10.0.100-dev"
    }
  ]
}

Todo:

tests

surayya-MS

Looks good to me! waiting for tests

src/Containers/Microsoft.NET.Build.Containers/ImageBuilder.cs

src/Containers/Microsoft.NET.Build.Containers/Tasks/CreateNewImage.cs

baronfel · 2025-01-17T20:40:26Z

Ok, added a test!

baronfel · 2025-01-20T16:29:43Z

/backport to release/8.0.4xx

baronfel · 2025-01-20T16:29:50Z

/backport to release/9.0.1xx

github-actions · 2025-01-20T16:29:54Z

Started backporting to release/8.0.4xx: https://github.com/dotnet/sdk/actions/runs/12872249580

github-actions · 2025-01-20T16:30:00Z

Started backporting to release/9.0.1xx: https://github.com/dotnet/sdk/actions/runs/12872251500

github-actions · 2025-01-20T16:30:14Z

@baronfel backporting to "release/8.0.4xx" failed, the patch most likely resulted in conflicts:

$ git am --3way --empty=keep --ignore-whitespace --keep-non-patch changes.patch

Applying: allow users to set a specific ContainerImageFormat to force OCI support
Using index info to reconstruct a base tree...
M	src/Containers/Microsoft.NET.Build.Containers/ContainerBuilder.cs
M	src/Containers/Microsoft.NET.Build.Containers/ImageBuilder.cs
A	src/Containers/Microsoft.NET.Build.Containers/PublicAPI/net10.0/PublicAPI.Unshipped.txt
M	src/Containers/Microsoft.NET.Build.Containers/PublicAPI/net472/PublicAPI.Unshipped.txt
M	src/Containers/Microsoft.NET.Build.Containers/Tasks/CreateNewImage.Interface.cs
M	src/Containers/Microsoft.NET.Build.Containers/Tasks/CreateNewImage.cs
M	src/Containers/Microsoft.NET.Build.Containers/Tasks/CreateNewImageToolTask.cs
M	src/Containers/containerize/ContainerizeCommand.cs
M	src/Containers/packaging/build/Microsoft.NET.Build.Containers.targets
Falling back to patching base and 3-way merge...
Auto-merging src/Containers/packaging/build/Microsoft.NET.Build.Containers.targets
CONFLICT (content): Merge conflict in src/Containers/packaging/build/Microsoft.NET.Build.Containers.targets
Auto-merging src/Containers/containerize/ContainerizeCommand.cs
CONFLICT (content): Merge conflict in src/Containers/containerize/ContainerizeCommand.cs
Auto-merging src/Containers/Microsoft.NET.Build.Containers/Tasks/CreateNewImageToolTask.cs
Auto-merging src/Containers/Microsoft.NET.Build.Containers/Tasks/CreateNewImage.cs
Auto-merging src/Containers/Microsoft.NET.Build.Containers/Tasks/CreateNewImage.Interface.cs
Auto-merging src/Containers/Microsoft.NET.Build.Containers/PublicAPI/net8.0/PublicAPI.Unshipped.txt
Auto-merging src/Containers/Microsoft.NET.Build.Containers/PublicAPI/net472/PublicAPI.Unshipped.txt
Auto-merging src/Containers/Microsoft.NET.Build.Containers/ImageBuilder.cs
Auto-merging src/Containers/Microsoft.NET.Build.Containers/ContainerBuilder.cs
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Patch failed at 0001 allow users to set a specific ContainerImageFormat to force OCI support
Error: The process '/usr/bin/git' failed with exit code 128

Please backport manually!

github-actions · 2025-01-20T16:30:33Z

@baronfel an error occurred while backporting to "release/9.0.1xx", please check the run log for details!

Server Error

baronfel · 2025-01-20T16:30:35Z

Backporting to 8.x and 9.x because this is requested by an internal team that has this constraint.

…rt (dotnet#46011)

allow users to set a specific ContainerImageFormat to force OCI support

5630083

dotnet-issue-labeler bot added Area-Containers Related to dotnet SDK containers functionality untriaged Request triage from a team member labels Jan 15, 2025

baronfel changed the title ~~allow users to set a specific ContainerImageFormat to force OCI support~~ Allow users to set a specific ContainerImageFormat to force OCI support Jan 15, 2025

baronfel requested a review from a team January 15, 2025 21:24

This was referenced Jan 16, 2025

The active test run was aborted. Reason: Test host process crashed dotnet/dnceng#451

Open

Failure on Helix tests - Tests timing out #40074

Open

surayya-MS reviewed Jan 16, 2025

View reviewed changes

src/Containers/Microsoft.NET.Build.Containers/ImageBuilder.cs Show resolved Hide resolved

src/Containers/Microsoft.NET.Build.Containers/ImageBuilder.cs Outdated Show resolved Hide resolved

src/Containers/Microsoft.NET.Build.Containers/Tasks/CreateNewImage.cs Show resolved Hide resolved

baronfel and others added 3 commits January 16, 2025 09:30

Update src/Containers/Microsoft.NET.Build.Containers/ImageBuilder.cs

f12b104

add error handling for invalid ImageFormat

2792e2b

add test to show forced base image

9493f15

baronfel mentioned this pull request Jan 18, 2025

Consider publishing .NET base images as OCI formats instead of Docker formats dotnet/dotnet-docker#6148

Open

surayya-MS approved these changes Jan 20, 2025

View reviewed changes

baronfel merged commit dc6d68f into dotnet:main Jan 20, 2025
35 of 38 checks passed

github-actions bot mentioned this pull request Jan 20, 2025

[release/9.0.1xx] Allow users to set a specific ContainerImageFormat to force OCI support #46138

Open

baronfel deleted the support-container-image-format-override branch January 20, 2025 16:30

baronfel added a commit to baronfel/sdk that referenced this pull request Jan 20, 2025

Allow users to set a specific ContainerImageFormat to force OCI suppo…

20890c9

…rt (dotnet#46011)

baronfel mentioned this pull request Jan 20, 2025

[release/8.0.4xx] Manual Backport of #40611 to 8.0.4xx - Allow users to set a specific ContainerImageFormat to force OCI support #46140

Open

baronfel added a commit to baronfel/sdk that referenced this pull request Jan 21, 2025

Allow users to set a specific ContainerImageFormat to force OCI suppo…

419231f

…rt (dotnet#46011)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow users to set a specific ContainerImageFormat to force OCI support #46011

Allow users to set a specific ContainerImageFormat to force OCI support #46011

baronfel commented Jan 15, 2025

surayya-MS left a comment

baronfel commented Jan 17, 2025

baronfel commented Jan 20, 2025

baronfel commented Jan 20, 2025

github-actions bot commented Jan 20, 2025

github-actions bot commented Jan 20, 2025

github-actions bot commented Jan 20, 2025

github-actions bot commented Jan 20, 2025

baronfel commented Jan 20, 2025

Allow users to set a specific ContainerImageFormat to force OCI support #46011

Allow users to set a specific ContainerImageFormat to force OCI support #46011

Conversation

baronfel commented Jan 15, 2025

surayya-MS left a comment

Choose a reason for hiding this comment

baronfel commented Jan 17, 2025

baronfel commented Jan 20, 2025

baronfel commented Jan 20, 2025

github-actions bot commented Jan 20, 2025

github-actions bot commented Jan 20, 2025

github-actions bot commented Jan 20, 2025

github-actions bot commented Jan 20, 2025

baronfel commented Jan 20, 2025