NativeAOT support in source-build

[crummel]

From dotnet/runtime#69611, there is more work here than just building llvm in source-build - we have to make some decisions about how different native packages are linked. Currently NativeAOT will not work in source-build because of the assumptions it makes about these linkages.

[MichaelSimons]

[Triage] one suggestion is to try an run it today w/the prebuilt to see how much is functional and what is broken.

[tmds]

Is NativeAOT a feature that gets delivered from the SDK, or does it ship out-of-band from nuget.org?

I've downloaded preview 7, and for NativeAOT I need a to add a reference to Microsoft.DotNet.ILCompiler.

If this ships out-of-band we can skip compiling it on source-build .NET.
Or do we still need to build a part? Or do we need it to build .NET itself?

cc @omajid

[tmds]

@jkotas can you answer my questions above?

[jkotas]

Is NativeAOT a feature that gets delivered from the SDK, or does it ship out-of-band from nuget.org?

If you specify <PublishAot>true</PublishAot> property in your .csproj, SDK will add native AOT package for you and the package gets downloaded from nuget during project restore.

It is same scheme as what's used for publishing self-contained apps (SDK adds Microsoft.NETCore.App.Runtime.* package reference) or PublishReadyToRun (SDK adds Microsoft.NETCore.App.Crossgen2.* package reference).

I've downloaded preview 7, and for NativeAOT I need a to add a reference to Microsoft.DotNet.ILCompiler.

Adding <PublishAot>true</PublishAot> property to your project file is going to be the recommended way to enable publishing as native aot. It should work in preview 7.

Just like with the crossgen2 package, you can manually add the native aot package reference and it will override the one added by the SDK. The explicit package reference should be only used as last resort (e.g. to workaround bugs in the runtime). It tends to produce version mismatches when people upgrade to new runtime, but forget to bump the aot package version.

The manual package overriding is broken in preview 7. It should be fixed in rc builds.

Or do we need it to build .NET itself?

It is not required to build .NET itself.

We use nativeaot to optimize crossgen2 where possible, but this optimization is disabled for source build currently: https://github.com/dotnet/runtime/blob/main/src/coreclr/tools/aot/crossgen2/crossgen2.csproj#L9

[tmds]

@jkotas thanks. I think this means we have no need to build NativeAOT support for source-build in .NET 7.

It is same scheme as what's used for publishing self-contained apps (SDK adds Microsoft.NETCore.App.Runtime.* package reference) or PublishReadyToRun (SDK adds Microsoft.NETCore.App.Crossgen2.* package reference).

Source-build doesn't need to build the end-user NativeAOT support.

Like self-contained apps in .NET 6, the SDK uses packages published by Microsoft on nuget.org.

I do think these should become features of source-built .NET (#1215). That's not for .NET 7.

It is not required to build .NET itself.

When we build .NET from source, we still need crossgen2 to perform R2R compilation on the runtime assemblies.

And crossgen2 gets built without NativeAOT to do this.

We use nativeaot to optimize crossgen2 where possible

This applies NativeAOT to crossgen2 application itself. We don't ship it.

This optimization doesn't improve the resulting runtime assemblies after R2R compilation (I assume).

[jkotas]

I think this means we have no need to build NativeAOT support for source-build in .NET 7.

I agree.

This optimization doesn't improve the resulting runtime assemblies after R2R compilation (I assume).

Correct.

[tmds]

If you specify true property in your .csproj, SDK will add native AOT package for you and the package gets downloaded from nuget during project restore.

@jkotas, Microsoft's .NET 7 preview 7 includes ./sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.DotNet.ILCompiler.

Does that mean it doesn't get downloaded from nuget, and we should build it from source?

[jkotas]

Microsoft's .NET 7 preview 7 includes ./sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.DotNet.ILCompiler

I have discovered that just a few days ago as well. Related problem is that we are bundling more stuff into the SDK than what should be required (dotnet/runtime#75468 is opened on that).

We either need to figure out how to remove the binaries from what we are bundling into SDK and/or we need to start building the package as part of the source build.

@LakshanF Could you please look into this?

[sbomer]

From my point of view, it would make sense to source-build the Microsoft.DotNet.ILCompiler package for 7.0. This would put us into a similar situation to crossgen, where the .target files are also bundled with the SDK (the difference is the crossgen targets actually live in SDK instead of flowing in from runtime).

I agree that we should eventually move the targets out of the bundled package to fix versioning issues, but I would hesitate to do so in 7.0 because the SDK doesn't have a way to import targets from the dynamically-downloaded "runtime packs" (the mechanism used for crossgen). We would need to work around this limitation and replace it with an implicit PackageReference, which could require changes to the targets logic. I'm happy to help in case we do want this to get fixed in 7.0.

[tmds]

Is it correct that the SDK doesn't actually contain the ILCompiler (that is: the thing that does nativeaot), but it contains parts of the ILCompiler package so the SDK knows how to get the actual ILCompiler from nuget.org?

[sbomer]

Yes, roughly. The Microsoft.DotNet.ILCompiler package doesn't include the ILCompiler executable. It has logic for locating the version of the actual compiler bits downloaded by the SDK from nuget. It also contains some MSBuild logic: tasks like ComputeManagedAssembliesToCompileToNative, and .targets that compute the command-line arguments to the compiler.

[crummel]

I've summarized what I believe to be the current state of play and the agreed-upon solution for NativeAOT in 7.0 in dotnet/runtime#76206. Please check this out and let me know if you have any concerns.

[MichaelSimons]

NativeAOT support removed from net 7.0 source-build with dotnet/installer#14697 - closing