Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Centralize on one container #60260

Merged
merged 1 commit into from
Feb 10, 2025
Merged

Centralize on one container #60260

merged 1 commit into from
Feb 10, 2025

Conversation

wtgodbe
Copy link
Member

@wtgodbe wtgodbe commented Feb 7, 2025

@wtgodbe wtgodbe requested a review from a team as a code owner February 7, 2025 19:16
@dotnet-issue-labeler dotnet-issue-labeler bot added the area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework label Feb 7, 2025
@wtgodbe wtgodbe mentioned this pull request Feb 7, 2025
@wtgodbe
Copy link
Member Author

wtgodbe commented Feb 8, 2025

/azp run

Copy link

Azure Pipelines successfully started running 3 pipeline(s).

@richlander
Copy link
Member

/azp run

Copy link

Azure Pipelines successfully started running 3 pipeline(s).

@richlander
Copy link
Member

Looks like we need awk. I'll add that. This is a bit tedious, but we will end up with the minimal set of dependencies we need. That's useful.

@richlander
Copy link
Member

@richlander
Copy link
Member

/azp run

Copy link

Azure Pipelines successfully started running 3 pipeline(s).

@richlander
Copy link
Member

/azp run

Copy link

Azure Pipelines successfully started running 3 pipeline(s).

@richlander
Copy link
Member

Looks like it worked, right @wtgodbe?

@richlander
Copy link
Member

Here's some perspective on this change.

$ docker images
REPOSITORY                                    TAG                                  IMAGE ID       CREATED        SIZE
mcr.microsoft.com/dotnet-buildtools/prereqs   azurelinux-3.0-net10.0-build-amd64   7018b8c69b65   5 hours ago    225MB
mcr.microsoft.com/dotnet-buildtools/prereqs   cbl-mariner-2.0-cross-arm64-alpine   c02b5140907b   5 days ago     5.26GB
mcr.microsoft.com/dotnet-buildtools/prereqs   cbl-mariner-2.0-cross-arm-alpine     6d12ebb8893b   5 days ago     5.14GB
mcr.microsoft.com/dotnet-buildtools/prereqs   alpine-3.19-WithNode                 6b5711daadc8   5 days ago     2.75GB
mcr.microsoft.com/dotnet-buildtools/prereqs   centos-stream8                       bdb4e45ffa7d   8 months ago   2.1GB

Those are uncompressed sizes, but I think the picture is pretty clear. 5GB! Wow. That's a lot bigger than I imagined. I was thinking 1.5GB territory.

@wtgodbe
Copy link
Member Author

wtgodbe commented Feb 10, 2025

Looks like it worked, right @wtgodbe?

Yep, looks good to me! The test failures appear unrelated. I'll re-kick the internal job just to be certain, but we should be good to merge this today (and get the 8/9 containers into 8.0 and 9.0 before the branches close EOD)

@wtgodbe
Copy link
Member Author

wtgodbe commented Feb 10, 2025

Ah, good thing we tried an internal build - generating the SAS token for the build failed with:

##[error]Azure CLI 2.x is not installed on this machine.
##[error]Script failed with error: Error: Unable to locate executable file: 'pwsh'. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable.

@richlander
Copy link
Member

richlander commented Feb 10, 2025

Here are the final (I hope) numbers.

In summary, we were using 4 large build images and are now using 1 smaller one. Runtime sets the model for building native code and now aspnetcore sets the model for building managed code. This, in effect, explains the massive difference in container size. aspnetcore was using the runtime container images and was not well-served by it since the need aspnetcore has is much more narrow.

{4E2C0D19-D07D-49D9-9937-E5FE01203779}

These numbers (all up) are not stable. I've switched machines and am using a Docker from different sources. It's primarily due (I'm guessing) to measuring with containerd (here) and not (earlier). The only thing that matters is the ratios.

The new image is still bigger than I imagined. Here's a quick test I ran that tells the story.

{5ED0F7BD-58CF-4A43-B857-737D486BBCE5}

Excuse the fact that the images are slightly out of order.

This tells us that the majority of our cost is from Azure CLI.

@wtgodbe
Copy link
Member Author

wtgodbe commented Feb 10, 2025

Internal build looks good, and test failure is unrelated - merging

@wtgodbe wtgodbe merged commit bb860a9 into main Feb 10, 2025
24 of 27 checks passed
@wtgodbe wtgodbe deleted the wtgodbe/DockerNew branch February 10, 2025 23:56
@dotnet-policy-service dotnet-policy-service bot added this to the 10.0-preview2 milestone Feb 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants