Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split the Runtime Shared Framework project and combine legs in the official build #111136

Merged
merged 45 commits into from
Jan 11, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
82f8304
Add new entrypoints for the Microsoft.NETCore.App.Runtime shared fram…
jkoritzinsky Jan 3, 2025
b701999
Make the "regular" runtime sfxproj into a coreclr-only sfxproj
jkoritzinsky Jan 3, 2025
8499495
Extract out much of the runtime-specific logic into the separate runt…
jkoritzinsky Jan 3, 2025
07446e0
Extract and share more logic with properties for the features instead…
jkoritzinsky Jan 3, 2025
308872d
Inline ReadyToRun.targets now that we're shipping all other tools as …
jkoritzinsky Jan 3, 2025
70103a4
Abstract out the static library selection.
jkoritzinsky Jan 3, 2025
b470ce7
Don't include a host-arch DiaSymReader.Native dll.
jkoritzinsky Jan 3, 2025
4abcb9a
Add new BuildAllRuntimePacks option to enable building runtime packs …
jkoritzinsky Jan 3, 2025
da8d371
Remove RuntimeFlavor from corehost build and make various adjustments…
jkoritzinsky Jan 3, 2025
734fb85
Don't build mono for linux-musl-arm(64) or windows-arm64
jkoritzinsky Jan 3, 2025
3b7b8eb
Combine jobs in the official build using our new properties
jkoritzinsky Jan 3, 2025
e18b717
Fix missing quote
jkoritzinsky Jan 3, 2025
e02e74e
Various fixes for different legs.
jkoritzinsky Jan 4, 2025
bf5b3bf
Fix template for Mono LLVM jobs
jkoritzinsky Jan 4, 2025
3784930
Remove now-unused template
jkoritzinsky Jan 4, 2025
880d0ee
Adjust when Crossgen2.sfxproj is built and try another variant for th…
jkoritzinsky Jan 6, 2025
f6baa37
Only build CoreCLR and Mono runtime packs when they're supported plat…
jkoritzinsky Jan 6, 2025
07e9ae9
Microsoft.NETCore.App.Runtime.sfxproj -> Microsoft.NETCore.App.Runtim…
jkoritzinsky Jan 6, 2025
1384947
Don't override RuntimeFlavor if it's been explicitly set (ie in the r…
jkoritzinsky Jan 6, 2025
3ba9654
Adjust condition for NativeAOT non-sfx-based packages
jkoritzinsky Jan 6, 2025
6f06a31
Fix installer pretest build
jkoritzinsky Jan 6, 2025
f7782ae
Remove now-unused runtime product and symbol archives (source-build n…
jkoritzinsky Jan 6, 2025
41f383b
Now that we don't have the separate "generate offsets" jobs, try coll…
jkoritzinsky Jan 6, 2025
74879aa
Re-collapse some of the YAML back together now that we've changed how…
jkoritzinsky Jan 6, 2025
a287651
Fix typo
jkoritzinsky Jan 6, 2025
08edb57
Add forwarding for DotNetBuildAllRuntimePacks -> BuildAllRuntimePacks…
jkoritzinsky Jan 6, 2025
bec0f55
Fix default subsets for android, apple mobile
jkoritzinsky Jan 7, 2025
39537a7
Fix bad comparison
jkoritzinsky Jan 7, 2025
6fbcaad
Adjust conditions
jkoritzinsky Jan 7, 2025
e16c531
Fix bad global property instead (that was the real issue)
jkoritzinsky Jan 7, 2025
e0e4c5a
Only generate installers for the projects we actually produce install…
jkoritzinsky Jan 7, 2025
f4d1c2e
Rename property
jkoritzinsky Jan 7, 2025
86873f4
Set OutDirName to avoid hitting MAX_PATH issues in the VMR
jkoritzinsky Jan 7, 2025
0d70d09
Rename installer pretest project
jkoritzinsky Jan 7, 2025
2180029
Merge branch 'main' of https://github.com/dotnet/runtime into split-r…
jkoritzinsky Jan 7, 2025
4ceddce
Adjust System.IO.Ports and host packages conditions to ensure we buil…
jkoritzinsky Jan 7, 2025
01f253a
Fix default runtime flavor for official builds where we build native …
jkoritzinsky Jan 8, 2025
2ab5771
Don't include the single file host in linux bionic
jkoritzinsky Jan 8, 2025
69c41ec
Build the host and the bundle on Linux bionic 64-bit
jkoritzinsky Jan 8, 2025
68dd8ef
Adjust the SkipBuild conditions
jkoritzinsky Jan 9, 2025
acad2cf
Merge branch 'split-runtime-sfxproj' of https://github.com/jkoritzins…
jkoritzinsky Jan 9, 2025
0f8f358
Adjust singlefilehost condition
jkoritzinsky Jan 9, 2025
ebef975
Merge branch 'main' of https://github.com/dotnet/runtime into split-r…
jkoritzinsky Jan 9, 2025
00852b2
Produce the nethost archives
jkoritzinsky Jan 10, 2025
0a8a89c
Merge branch 'main' of https://github.com/dotnet/runtime into split-r…
jkoritzinsky Jan 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions eng/DotNetBuild.props
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@
<InnerBuildArgs Condition="'$(DotNetBuildMonoAOTEnableLLVM)' != ''">$(InnerBuildArgs) /p:MonoAOTEnableLLVM=$(DotNetBuildMonoAOTEnableLLVM)</InnerBuildArgs>
<InnerBuildArgs Condition="'$(DotNetBuildMonoBundleLLVMOptimizer)' != ''">$(InnerBuildArgs) /p:MonoBundleLLVMOptimizer=$(DotNetBuildMonoBundleLLVMOptimizer)</InnerBuildArgs>
<InnerBuildArgs Condition="'$(DotNetBuildMonoCrossAOT)' != ''">$(InnerBuildArgs) /p:DotNetBuildMonoCrossAOT=$(DotNetBuildMonoCrossAOT)</InnerBuildArgs>
<InnerBuildArgs Condition="'$(DotNetBuildAllRuntimePacks)' != ''">$(InnerBuildArgs) /p:DotNetBuildAllRuntimePacks=$(DotNetBuildAllRuntimePacks)</InnerBuildArgs>
<InnerBuildArgs Condition="'$(PgoInstrument)' == 'true'">$(InnerBuildArgs) $(FlagParameterPrefix)pgoinstrument</InnerBuildArgs>

<!-- This prop needs to be passed to the inner build manually as the BaseInnerSourceBuildCommand gets overridden above -->
Expand Down
4 changes: 2 additions & 2 deletions eng/Publishing.props
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@
<Target Name="GetNonStableProductVersion">
<!-- Retrieve the non-stable runtime pack product version.
Don't stabilize the package version in order to retrieve the VersionSuffix. -->
<MSBuild Projects="$(RepoRoot)src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj"
<MSBuild Projects="$(RepoRoot)src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.CoreCLR.sfxproj"
Targets="ReturnProductVersion"
Properties="IsShipping=false;
Crossgen2SdkOverridePropsPath=;
Expand All @@ -165,7 +165,7 @@
BeforeTargets="PublishToAzureDevOpsArtifacts"
Condition="'$(_ShouldGenerateProductVersionFiles)' == 'true'">
<!-- Retrieve the runtime pack product version. -->
<MSBuild Projects="$(RepoRoot)src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj"
<MSBuild Projects="$(RepoRoot)src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.CoreCLR.sfxproj"
Targets="ReturnProductVersion"
Properties="Crossgen2SdkOverridePropsPath=;
Crossgen2SdkOverrideTargetsPath=">
Expand Down
139 changes: 98 additions & 41 deletions eng/Subsets.props

Large diffs are not rendered by default.

13 changes: 0 additions & 13 deletions eng/pipelines/common/platform-matrix-multijob.yml

This file was deleted.

284 changes: 101 additions & 183 deletions eng/pipelines/runtime-official.yml

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion eng/testing/workloads-browser.targets
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@
<!-- We need nugets for all wasm runtime flavors. The one corresponding the current
property values is already built, the others need to be added to NuGetsToBuildForWorkloadTesting -->
<NuGetsToBuildForWorkloadTesting Include="$(_DefaultRuntimePackNuGetPath)"
Project="$(InstallerProjectRoot)pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.sfxproj"
Project="$(InstallerProjectRoot)pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.Mono.sfxproj"
Dependencies="$(_DefaultRuntimePackNuGetPath)"
Properties="@(_DefaultPropsForNuGetBuild, ';');WasmEnableThreads=$(_IsMTNugetMissing)"
Descriptor="$(_Descriptor)"/>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
<NativeBinary Include="$(DotNetHostBinDir)/hostfxr.h" />
<NativeBinary Include="$(DotNetHostBinDir)/nethost.h" />
</ItemGroup>
<ItemGroup Condition="'$(RuntimeFlavor)' != 'Mono'">
<ItemGroup Condition="'$(RuntimeFlavor)' != 'Mono' and '$(TargetsLinuxBionic)' != 'true'">
<NativeBinary Include="$(DotNetHostBinDir)/singlefilehost$(ExeSuffix)" />
</ItemGroup>
<ItemGroup Condition="'$(TargetOS)' == 'windows'">
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<Sdk Name="Microsoft.DotNet.SharedFramework.Sdk" />

<PropertyGroup>
<SkipBuild Condition="'$(RuntimeFlavor)' != '$(PrimaryRuntimeFlavor)' or '$(TargetsMobile)' == 'true'">true</SkipBuild>
<SkipBuild Condition="'$(DotNetBuildAllRuntimePacks)' != 'true' and ('$(RuntimeFlavor)' != '$(PrimaryRuntimeFlavor)' or '$(TargetsMobile)' == 'true')">true</SkipBuild>
<PlatformPackageType>AppHostPack</PlatformPackageType>
<UseTemplatedPlatformManifest>true</UseTemplatedPlatformManifest>
<ArchiveName>dotnet-apphost-pack</ArchiveName>
Expand Down Expand Up @@ -31,7 +31,7 @@
<NativeRuntimeAsset Include="$(DotNetHostBinDir)/hostfxr.h" />
<NativeRuntimeAsset Include="$(DotNetHostBinDir)/coreclr_delegates.h" />
</ItemGroup>
<ItemGroup Condition="'$(RuntimeFlavor)' != 'Mono'">
<ItemGroup Condition="'$(RuntimeFlavor)' != 'Mono' and '$(TargetsLinuxBionic)' != 'true'">
<NativeRuntimeAsset Include="$(DotNetHostBinDir)/singlefilehost$(ExeSuffix)" />
</ItemGroup>
<ItemGroup Condition="'$(TargetOS)' == 'windows'">
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
<PermitDllAndExeFilesLackingFileVersion>true</PermitDllAndExeFilesLackingFileVersion>
<AotCompilerFileName>mono-aot-cross</AotCompilerFileName>
<AotCompilerFileName Condition="$([MSBuild]::IsOsPlatform('Windows'))">$(AotCompilerFileName).exe</AotCompilerFileName>
<SkipInstallersPackageReference>true</SkipInstallersPackageReference>
</PropertyGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
<Project TreatAsLocalProperty="RuntimeFlavor;BuildNativeAOTRuntimePack">
<PropertyGroup>
<RuntimeFlavor>CoreCLR</RuntimeFlavor>
<BuildNativeAOTRuntimePack>false</BuildNativeAOTRuntimePack>
<OutDirName>coreclr-pack</OutDirName>
</PropertyGroup>
<Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" />
<Import Project="Sdk.props" Sdk="Microsoft.DotNet.SharedFramework.Sdk" />

<PropertyGroup>
<PlatformPackageType>RuntimePack</PlatformPackageType>
<InstallerName Condition="'$(TargetOS)' != 'osx'">dotnet-runtime</InstallerName>
<InstallerName Condition="'$(TargetOS)' == 'osx'">dotnet-runtime-internal</InstallerName>
<IsPackable Condition="'$(BuildOnlyPgoInstrumentedAssets)' != ''">false</IsPackable>
<VSInsertionShortComponentName>NetCore.SharedFramework</VSInsertionShortComponentName>
<UseTemplatedPlatformManifest>true</UseTemplatedPlatformManifest>
<UseRidGraph>true</UseRidGraph>
</PropertyGroup>

<ItemGroup>
<LinuxPackageDependency Include="dotnet-hostfxr-$(MajorVersion).$(MinorVersion);dotnet-runtime-deps-$(MajorVersion).$(MinorVersion)" Version="$(InstallerPackageVersion)" />
</ItemGroup>

<PropertyGroup>
<PublishReadyToRun>true</PublishReadyToRun>
<!-- Disable crossgen on NetBSD, illumos, Solaris and Haiku for now. This can be revisited when we have full support. -->
<PublishReadyToRun Condition="'$(TargetOS)' == 'netbsd' or '$(TargetOS)' == 'illumos' or '$(TargetOS)' == 'solaris' or '$(TargetOS)' == 'haiku'">false</PublishReadyToRun>
<!-- Disable crossgen on FreeBSD when cross building from Linux. -->
<PublishReadyToRun Condition="'$(TargetOS)'=='freebsd' and '$(CrossBuild)'=='true'">false</PublishReadyToRun>
<UsesMuxerHostModel Condition="'$(TargetsMobile)' != 'true'">true</UsesMuxerHostModel>
<IncludeDiaSymReaderNative Condition="'$(TargetOS)' == 'windows'">true</IncludeDiaSymReaderNative>
<UseLocalCrossgen2Pack>$(PublishReadyToRun)</UseLocalCrossgen2Pack>
</PropertyGroup>

<Import Project="Microsoft.NETCore.App.Runtime.props" />

<PropertyGroup>
<PublishReadyToRunComposite Condition="$(ForcePublishReadyToRunComposite) == 'true'">true</PublishReadyToRunComposite>
</PropertyGroup>


<Target Name="ResolveReadyToRunCompilers" DependsOnTargets="ResolveRuntimeFilesFromLocalBuild">
<PropertyGroup>
<Crossgen2Path>$([MSBuild]::NormalizePath('$(Crossgen2InBuildDir)', 'crossgen2$(ExeSuffix)'))</Crossgen2Path>
</PropertyGroup>

<ItemGroup>
<Crossgen2Tool Include="$(Crossgen2Path)"
TargetArch="$(TargetArchitecture)"
TargetOS="$(TargetOS)"
PerfmapFormatVersion="$(PublishReadyToRunPerfmapFormatVersion)"/>
</ItemGroup>
</Target>

<!--
Put the mibc file into tools and not into PgoData. Add it after GetFilesToPackage to ensure it isn't automatically classified.
Putting it in tools will also hide it from being part of the RuntimeList.xml.
-->
<Target Name="AddMibcFilesToPackage"
AfterTargets="GetFilesToPackage">
<ItemGroup>
<FilesToPackageMibcData Include="$(CoreCLRArtifactsPath)StandardOptimizationData.mibc"
Condition="Exists('$(CoreCLRArtifactsPath)StandardOptimizationData.mibc')">
<TargetPath>tools</TargetPath>
</FilesToPackageMibcData>
<FilesToPackage Include="@(FilesToPackageMibcData)"/>
</ItemGroup>
</Target>


<Target Name="AddCoreCLRRuntimeFiles" AfterTargets="AddRuntimeFilesToPackage">
<ItemGroup>
<CoreCLRCrossTargetFiles PackOnly="true" />
<CoreCLRCrossTargetFiles Condition="$([System.String]::new('%(FileName)').StartsWith('mscordaccore')) and '$(TargetsWindows)' == 'true'">
<TargetPath>tools/$(CoreCLRCrossTargetComponentDirName)_$(TargetArchitecture)</TargetPath>
</CoreCLRCrossTargetFiles>
<CoreCLRCrossTargetFiles Condition="'%(FileName)%(Extension)' == 'mscordbi.dll' and '$(TargetsWindows)' == 'true'">
<TargetPath>tools/$(CoreCLRCrossTargetComponentDirName)_$(TargetArchitecture)</TargetPath>
</CoreCLRCrossTargetFiles>

<ReferenceCopyLocalPaths Include="@(CoreCLRCrossTargetFiles)" />
</ItemGroup>
</Target>
</Project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
<Project TreatAsLocalProperty="RuntimeFlavor;BuildNativeAOTRuntimePack">
<PropertyGroup>
<RuntimeFlavor>Mono</RuntimeFlavor>
<BuildNativeAOTRuntimePack>false</BuildNativeAOTRuntimePack>
<OutDirName>mono-pack</OutDirName>
</PropertyGroup>

<Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" />
<Import Project="Sdk.props" Sdk="Microsoft.DotNet.SharedFramework.Sdk" />

<PropertyGroup>
<PlatformPackageType>RuntimePack</PlatformPackageType>
<UseTemplatedPlatformManifest>true</UseTemplatedPlatformManifest>
<UseRidGraph>true</UseRidGraph>
</PropertyGroup>

<PropertyGroup>
<SkipInstallerBuild>true</SkipInstallerBuild>
</PropertyGroup>

<PropertyGroup>
<PublishReadyToRun>false</PublishReadyToRun>
<UsesMuxerHostModel Condition="'$(TargetsMobile)' != 'true'">true</UsesMuxerHostModel>
<IncludeDiaSymReaderNative Condition="'$(TargetOS)' == 'windows'">true</IncludeDiaSymReaderNative>
<IncludeStaticLibrariesInPack Condition="'$(TargetsMobile)' == 'true'">true</IncludeStaticLibrariesInPack>
</PropertyGroup>

<PropertyGroup Condition="'$(RuntimeFlavor)' != '$(PrimaryRuntimeFlavor)' or '$(TargetsMobile)' == 'true'">
<RuntimeSpecificFrameworkSuffix>Mono</RuntimeSpecificFrameworkSuffix>
<IsShipping Condition="'$(TargetsMobile)' != 'true'">false</IsShipping>
</PropertyGroup>
<PropertyGroup Condition="'$(MonoEnableLLVM)' == 'true' and '$(TargetsMobile)' != 'true'">
<RuntimeSpecificFrameworkSuffix>Mono.LLVM</RuntimeSpecificFrameworkSuffix>
<IsShipping>false</IsShipping>
</PropertyGroup>
<PropertyGroup Condition="'$(MonoAOTEnableLLVM)' == 'true' and '$(TargetsMobile)' != 'true'">
<RuntimeSpecificFrameworkSuffix>Mono.LLVM.AOT</RuntimeSpecificFrameworkSuffix>
<IsShipping>false</IsShipping>
</PropertyGroup>
<PropertyGroup Condition="'$(WasmEnableThreads)' == 'true'">
<RuntimeSpecificFrameworkSuffix>Mono.multithread</RuntimeSpecificFrameworkSuffix>
</PropertyGroup>
<PropertyGroup Condition="'$(RuntimeSpecificFrameworkSuffix)' != ''">
<OverridePackageId>$(SharedFrameworkName).Runtime.$(RuntimeSpecificFrameworkSuffix).$(RuntimeIdentifier)</OverridePackageId>
</PropertyGroup>

<Import Project="Microsoft.NETCore.App.Runtime.props" />

<Target Name="AddMonoRuntimeFiles" AfterTargets="AddRuntimeFilesToPackage">
<ItemGroup>
<MonoRuntimeFiles Include="@(MonoFrameworkReleaseFiles)"
ExcludeFromDataFiles="true">
<TargetPath>runtimes/$(RuntimeIdentifier)/native/Mono.release.framework/%(RecursiveDir)</TargetPath>
</MonoRuntimeFiles>

<MonoRuntimeFiles Include="@(MonoFrameworkDebugFiles)"
ExcludeFromDataFiles="true">
<TargetPath>runtimes/$(RuntimeIdentifier)/native/Mono.debug.framework/%(RecursiveDir)</TargetPath>
</MonoRuntimeFiles>

<MonoRuntimeFiles Include="@(MonoIncludeFiles)"
ExcludeFromDataFiles="true">
<TargetPath>runtimes/$(RuntimeIdentifier)/native/include/%(RecursiveDir)</TargetPath>
</MonoRuntimeFiles>

<MonoRuntimeFiles Include="@(MonoBuildFiles)"
ExcludeFromDataFiles="true">
<TargetPath>runtimes/$(RuntimeIdentifier)/build/%(RecursiveDir)</TargetPath>
</MonoRuntimeFiles>
<ReferenceCopyLocalPaths Include="@(MonoRuntimeFiles)" />
</ItemGroup>
</Target>
</Project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
<Project TreatAsLocalProperty="RuntimeFlavor;BuildNativeAOTRuntimePack">
<PropertyGroup>
<RuntimeFlavor>CoreCLR</RuntimeFlavor>
<BuildNativeAOTRuntimePack>true</BuildNativeAOTRuntimePack>
<OutDirName>naot-pack</OutDirName>
</PropertyGroup>

<Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" />
<Import Project="Sdk.props" Sdk="Microsoft.DotNet.SharedFramework.Sdk" />

<PropertyGroup>
<PlatformPackageType>RuntimePack</PlatformPackageType>
<UseTemplatedPlatformManifest>true</UseTemplatedPlatformManifest>
<UseRidGraph>true</UseRidGraph>
<SkipInstallerBuild>true</SkipInstallerBuild>
<PublishReadyToRun>false</PublishReadyToRun>
<OverridePackageId>$(SharedFrameworkName).Runtime.NativeAOT.$(RuntimeIdentifier)</OverridePackageId>
<IncludeStaticLibrariesInPack>true</IncludeStaticLibrariesInPack>
</PropertyGroup>

<ItemGroup>
<IgnoredDuplicateType Include="Internal.Runtime.CompilerHelpers.LibraryInitializer" />
</ItemGroup>

<Import Project="Microsoft.NETCore.App.Runtime.props" />

<Target Name="IncludeSymbolFilesInNativeAOTPackage"
AfterTargets="GetFilesToPackage">
<ItemGroup>
<!-- ILC needs the symbols next to the binaries -->
<FilesToPackage Update="@(FilesToPackage->WithMetadataValue('IsSymbolFile', 'true'))">
<IncludeAlways>true</IncludeAlways>
</FilesToPackage>
</ItemGroup>
</Target>
</Project>
Loading
Loading