Add Dockerfiles for Linux .NET Composite Images #4343
Comments
|
@mangod9 @richlander Here's the issue we can use to track progress and discussions to get the .NET composite images, and Dockerfiles, published to customers. |
|
Adding @trylek as well as he has been a key contributor to this project. |
Is a .NET composite a single file that bundles parts of different assemblies?
What causes it to start faster? and achieve a better performance? |
|
The .NET composite image bundles together the ready-to-run native code from all component assemblies. The managed assemblies themselves remain on disk but they are basically pure MSIL. (Everything can then be bundled into one file using the Single EXE .NET feature though.) The better startup and to a certain extent performance (so far we've mostly been focusing on startup) stems from the fact that a composite image can be treated as a "versioning bubble". The limitation that it always naturally needs to be shipped / serviced as a whole lets us do the following:
So far we haven't been focusing too much on steady state performance as that's supposed to be optimized by runtime tiered JITting. The problem with steady state performance is that many perf-critical constructs in general JIT and in the framework foundations e.g. cleaning up local variable area on the stack in method prologs or string manipulations can be much better optimized in the presence of various instruction set extensions (e.g. AVX2 on x64) while by default we strive to generate code compatible with all CPU variants i.e. using only the base instruction set. Crossgen2 does include options to declare what instruction set extensions are guaranteed to be present at runtime and in their presence it can generate the better optimized code right away but we tend to see that as a niche scenario that's only worth the effort where super high performance is of bigger value than compatibility. |
|
If I understand correctly, the composite image is a single file that provides the functionality otherwise split in many shared framework dlls in an optimized way? Are we building these composites as part of source-build? Are they included in the shared framework folders (like |
Yes, the artifact that we'll use for the Docker image already includes everything built. In other words, the Dockerfile's code here won't look very different from say the normal existing Runtime one.
That's right. |
|
The composite Dockerfile would look as follows: ARG REPO=mcr.microsoft.com/dotnet/runtime-deps
# Installer image
FROM amd64/buildpack-deps:jammy-curl AS installer
# Retrieve Composite .NET Runtime
RUN dotnet_version=7.0.2 \
&& curl -fSL --output dotnet.tar.gz https://dotnetcli.azureedge.net/dotnet/aspnetcore/Runtime/$dotnet_version/aspnetcore-runtime-$dotnet_version-linux-x64-composite.tar.gz \
&& mkdir -p /dotnet \
&& tar -oxzf dotnet.tar.gz -C /dotnet \
&& rm dotnet.tar.gz
# .NET Runtime Base Image
FROM $REPO:7.0.2-jammy-amd64
# .NET Runtime and ASP.NET Versions
ENV DOTNET_VERSION=7.0.2
ENV ASPNET_VERSION=7.0.2
COPY --from=installer ["/dotnet", "/usr/share/dotnet"]
RUN ln -s /usr/share/dotnet/dotnet /usr/bin/dotnetHere it's using .NET 7 for testing, but the purpose is to release it for .NET 8. |
|
I was thinking about the layering. We were talking about: I am guessing we'll be getting layer sharing (in an absolute byte sense) by: We should be able to determine this through some quick measurements. I'm guessing that the |
Today, the SDK Dockerfile includes only the SDK "stuff", implicitly filtering out the runtime and ASP.NET Core files. You're suggesting that the SDK Dockerfile now include the ASP.NET Core files in that logic and be based directly on the |
|
[Triage] For the SDK, we should favor ASP.NET Core scenarios and have the SDK based on aspnet. There's a tradeoff associated with this because someone that would need to pull the runtime image would have to reacquire those bits. There are likely to be more cases where people are accessing the aspnet image than the runtime image so it makes more sense to favor aspnet. flowchart TD
A[runtime-deps] --> B[runtime]
A --> C[aspnet]
C --> D[sdk]
|
|
This test may be affected and need to compensate for this change: |
|
I did some work on this and had a few findings:
On the bright side, we should have coherent runtime versions for previews, servicing releases, and GA. Just brainstorming some potential workarounds from the dotnet-docker side:
From outside the dotnet-docker space:
An ideal solution would be able to avoid the hard coherency requirement and build the images all the time for testing. |
|
I have spent some time looking into this. The first problem I see is that it seems to me that the It would be great if someone on @dotnet/crossgen-contrib (like @davidwrighton who I believe originally implemented parts of this logic) could confirm my findings but for now I believe we must primarily resolve this, I think it's quite likely that this inconsistency is the root of the problem that Logan and Matt are hitting. Wherever the full composite is put, it needs to be next to all the ASP.NET and framework assemblies because these get rewritten in the compilation process to acquire the forwarding header to the composite image. Either we put it in |
|
I wonder if the composite layout is somehow flawed why doesnt the issue repro consistently for the asp.net image? From what @lbussell describes it only happens when applied to the sdk image? |
|
So, this isn't something I built, but I can lay out what we need to do.
|
|
Well, for the non-composite ASP.NET image it doesn't matter, the framework and ASP.NET assemblies are separate and each is put in their respective folders. It's only the combined composite that ties these two groups together. |
|
@davidwrighton - Thanks for your feedback, so are you saying that having the |
|
@ivdiazsa can confirm but I believe the work that was done in asp.net repo did what @davidwrighton lists above. Are you noticing that the update asp.net and framework assemblies are not copied over? |
|
Hmm, I think you're right, I probably messed up the folder names when analyzing the different tarball flavors, I continue looking what's wrong with the Dockerfile then. |
|
@lbussell - I have uploaded your Dockerfile with a bunch of my fixes here: https://gist.github.com/trylek/76e2a0a146fcee9391302473033c888d It seems to me that it lets me build the container just fine, could you please give it a shot? I'll be happy to elaborate on each of the changes I've made. |
|
This is the original script Logan shared with me: https://gist.github.com/lbussell/45f157e1f7beaec0f3e073793a52b1f3 |
|
@trylek - What is the scope of your proposal? Is this a workaround or something we would ship? The changes in the aspnet-composite which overrides the Microsoft.NETCore.App from the runtime image isn't something we would want to ship. This would negate any size wins the composite provides. |
|
Hmm, I'm not sure I'm getting this. I believe the overall purpose of this entire effort is to optimize startup by using the combined ASP.NET+runtime composite coming from the aspnetcore repo. It's indeed what I'm expecting to be shipped at some point. What am I missing? |
|
@MichaelSimons - Just as a caveat, one somewhat confusing aspect of the script, probably caused by a series of copy & paste, is that multiple tarballs get downloaded under the name |
|
Other than that, for tracking and discussing changes it would be ideal to make something like a PR we could use to discuss changes to the Dockerfile; in practice, as I understood from Ivan, the actual Dockerfile is getting generated from some meta-format, I'm not yet sufficiently familiar with the related logic to see whether it would be helpful to discuss these changes directly in context of the meta-dockerfile, ideally directly as a PR against the |
|
The following line in the aspnet-composite layer raises red flags to me. COPY --from=aspnet-composite-installer ["/shared/Microsoft.NETCore.App", "/usr/share/dotnet/shared/Microsoft.NETCore.App"] What are you expecting this to do assuming we have a coherent build? |
I think it would be helpful to have a draft PR to review. We don't have to worry about the templating infra at this point. We could just modify one slice of Dockerfiles - e.g the 8.0 bookworm amd64 Dockerfiles (runtime-deps, runtime, aspnet, and sdk). Once we have an agreed upon Dockerfile(s), @lbussell can update the templates accordingly and regenerate the full set of impacted Dockerfiles. |
|
@trylek Thanks, I've taken a look as well. For the most part I agree with Michael. I would be concerned with duplicating the Runtime bits or having two versions of Microsoft.NETCore.App if we layer the aspnet composite image on top of the runtime image. Apologies for creating a confusingly large Dockerfile, I was trying to structure it in a way that you can get a one-line repro of the issue, without multiple Dockerfiles and a script. I will put together a draft PR that will make the changes easier to reason about. |
This is a leading question 😃. If I understand what is happening here correctly, the aspnet-composite image is carrying multiple copies of the shared framework because Docker utilizes a union file system. This is something we would not want to do and I postulate would negate the value of using composite images. |
|
Well, according to my understanding we pull down the combined ASP.NET+runtime composite image and use it to populate the |
|
My current understanding is that, in the absence of a coherent build, we end up with two versions of |
|
Another way to approach this would be to put [lib]coreclr.[dll|so] and such directly into the "dotnet" app folder, that would make "dotnet" completely self-contained. I have no idea why we're not doing it this way. |
Yes I understand that but because the individual assemblies need to be updated to their rewritten versions containing the forwarding ReadyToRun header with the name of the composite image and the Docker union file system, the image is still carrying multiple copies. Because of this, the aspnet-composite image cannot be based on the runtime image. |
|
Hmm, perhaps I don't understand Docker sufficiently to answer this. Semantically the aspnet-composite image includes its own version of the runtime framework so if I was building such a thing locally, I would either skip the runtime installation or let aspnet-composite installation stomp over it. I defer to more qualified experts how to express this behavior in terms of the Dockerfile. |
|
I opened the draft PR at #4532. |
With a union file system, you can have the semantics of stomping but your image layers will contain the original files that were "stomped". I'll try to explain this. First thing to note is that Dockerfile instructions equate to image layers. Layer 1 Layer 2 When you pull an image with these two layers, you will pull the entire contents of each indivedual layer which is: When the layers are extracted on disk, the resulting filesystem will contain: If small image sizes are a goal with your Docker image in order to achieve quick cold startups, you want to avoid situations where you ship multiple versions of files. |
|
Please see #4538 (comment) for the latest updates on the experimentation with producing the composite tarball out of the installer repo. |
|
This has been merged to main and published with the 8.0 Preview 5 release. |
and others
Describe the Problem
We have been working on .NET composites where, by bundling together the most needed assemblies, we are able to deliver a lower startup time, and overall better performance when running .NET apps. Our next step is to have the official Docker images recurrently built, alongside the existing .NET Linux images, so that customers can consume them.
Describe the Solution
We currently build them by hand but that can only be used for internal testing. We would like to have them officially published in the website: https://hub.docker.com/_/microsoft-dotnet
Additional Context
In order to achieve this purpose, we would like to add the corresponding Dockerfiles in this repo, which will pull the composites from the ASP.NET repo artifacts, and use them in a similar fashion as the currently existing Dockerfiles.
The text was updated successfully, but these errors were encountered: