GenerateRuntimeGraph fails when building runtime in source-build mode on Alpine #84127
Comments
dotnet-issue-labeler
bot
added
the
needs-area-label
ghost
added
the
omajid
changed the title
omajid
changed the title
|
This is probably due to #77508 being merged in 6.0 for runtime, but not the changes in installer: dotnet/installer#14816. The fix is to revert #77508 and maybe some other PRs that got merged for eliminating the portable build with 6.0. |
Reverting this particular one would break a workaround we use for #73525 as TargetRid would never be passed to __DistroRid. I'm looking into this right now to propose a better fix. |
The fix for this can backported. |
Could you point to the fix for this so I can backport? |
I am not sure I follow. I ran into this issue when building dotnet/runtime standalone. How is dotnet/installer involved? |
If you want to make it work standalone on 6.0 as-is, you can add
|
Applying the following diff yields the same behaviour, as well: diff --git a/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj b/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj
index 84d4bb88604..9fc8774c083 100644
--- a/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj
+++ b/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj
@@ -18,7 +18,7 @@
<PackageDescription>Provides runtime information required to resolve target framework, platform, and runtime specific implementations of .NETCore packages.</PackageDescription>
<!-- When building from source, ensure the RID we're building for is part of the RID graph -->
- <AdditionalRuntimeIdentifiers Condition="'$(DotNetBuildFromSource)' == 'true'">$(AdditionalRuntimeIdentifiers);$(OutputRID)</AdditionalRuntimeIdentifiers>
+ <AdditionalRuntimeIdentifiers Condition="'$(DotNetBuildFromSource)' == 'true'">linux-musl-x64</AdditionalRuntimeIdentifiers>
<ServicingVersion>8</ServicingVersion>
<GeneratePackageOnBuild>true</GeneratePacm kageOnBuild>
</PropertyGroup>Thus, this seems to have to do something with how GenerateRuntimeGraph parses AdditionalRuntimeIdentifiers. If it had to do with how OutputRid was set (or not, in the case of lack of the installer backport), this would've fixed the error. Also, re-reading omajid's tests, they also tested against
Awesome, thank you for pointing that out :) |
There were other PRs backported to 6.0 for eliminating portable build, like #77510. I think we should start by reverting those, and seeing if there is still an issue to be solved.
afaik everything works well on |
vcsjones
added
area-Infrastructure
and removed
needs-area-label
|
Tagging subscribers to this area: @dotnet/runtime-infrastructure Issue DetailsDescriptionThe dotnet/runtime build fails when running on Alpine in source-build mode with a portable configuration: The problem is most visible with .NET 6, because dotnet/runtime is built in portable mode as part of the end-to-end source-build process. With .NET 7 and .NET 8, source-build only builds dotnet/runtime in non-portable mode, so this problem doesn't appear there out of the box. It only appears when building dotnet/runtime by itself, in a configuration that's not used by source-build. Reproduction StepsCreate an alpine container using a # Dockerfile suitable for building .NET on Alpine
FROM alpine:3.17
RUN apk update && \
apk add \
bash \
binutils \
clang \
cmake \
git \
gcc \
icu-dev \
krb5-dev \
llvm \
libstdc++ \
linux-headers \
lttng-ust-dev \
make \
musl-dev \
openssl-dev \
python3 \
zlib-dev \Then main: release/7.0 and release/6.0 Expected behaviorBuild works Actual behaviorBuild fails Regression?No response Known WorkaroundsNo response ConfigurationNo response Other informationNo response
|
On My goal was to build runtime in portable mode. Is it supposed to work with just |
I'm not sure if you actually need to set something because there is Alpine handling here: Line 198 in e579ccb If it's needed, you need to set Lines 34 to 35 in e579ccb
I'm not sure what you want to build. source-build ( |
|
I can reproduce this on .NET 7.0.104 and on .NET 8.0.0-preview.2. This referenced jobs build runtime in portable mode while setting I still suspect there's an issue with the code. When error code occurs by: ,${rid} equals x64 when AdditionalRuntimeId is set to linux-musl-x64. Maybe something gnarly occurs in the parsing logics here?runtime/src/libraries/Microsoft.NETCore.Platforms/src/RID.cs Lines 59 to 122 in e13f0dc Note: I needed to force OutputRid to linux-musl-x64 on dotnet8, else the build would target alpine.3.17-x64. I suspect the parsing logics fizz out due to m̀ultiple dashes. I'm C# stupid, so can't help more other than vague intuitions. |
|
To test this hypothesis, I attempted a build with OutputRid set as linuxmusl-x64. Of course, it failed as no parent as defined. But it failed with this error: |
What command are you running? on what branch of what repo? |
|
Tag |
|
I extracted the RID parsing code, and passed https://dotnetfiddle.net/3ej5qT As you can see, A quick fix is to skip adding diff --git a/src/runtime/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj b/src/runtime/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platfor
ms.csproj
index 1df893388..6a1591035 100644
--- a/src/runtime/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj
+++ b/src/runtime/src/libraries/Microsoft.NETCore.Platforms/src/Microsoft.NETCore.Platforms.csproj
@@ -23,7 +23,9 @@
<_generateRuntimeGraphTargetFramework Condition="'$(MSBuildRuntimeType)' != 'core'">net472</_generateRuntimeGraphTargetFramework>
<_generateRuntimeGraphTask>$([MSBuild]::NormalizePath('$(BaseOutputPath)', $(Configuration), '$(_generateRuntimeGraphTargetFramework)', '$(AssemblyName).dll'))</_generateRuntimeGraphTask>
<!-- When building from source, ensure the RID we're building for is part of the RID graph -->
- <AdditionalRuntimeIdentifiers Condition="'$(DotNetBuildFromSource)' == 'true'">$(AdditionalRuntimeIdentifiers);$(OutputRID)</AdditionalRuntimeIdentifiers>
+ <!-- Skips 'linux-musl' and 'linux-bionic' as 'RID.Parse' doesn't know how to parse 'BaseRID' containing a '-' character
+ See https://github.com/dotnet/runtime/issues/84127 -->
+ <AdditionalRuntimeIdentifiers Condition="'$(DotNetBuildFromSource)' == 'true' AND ( '$(RuntimeOS)' != 'linux-musl' OR '$(RuntimeOS)' != 'linux-bionic' )">$(AdditionalRuntimeIdentifiers);$(OutputRid)</AdditionalRuntimeIdentifiers>
</PropertyGroup>
<ItemGroup Condition="'$(TargetFrameworkIdentifier)' == '.NETFramework'">An actual fix is adding in a case for parsing |
Aha, good you found the root cause! Looking at the code here: runtime/src/libraries/Microsoft.NETCore.Platforms/src/RID.cs Lines 91 to 95 in 042a350 We should add something like the following (at line 93), which causes the first // ensure there's no architeture later in the string
if (runtimeIdentifier.IndexOf(ArchitectureDelimiter, i + 1) != -1)
{
break;
} |
Nice, that did it! I wish I could read C#! Want me to get a PR going for this? |
I was going to suggest adding something to the tests, and looking at them I see: runtime/src/libraries/Microsoft.NETCore.Platforms/tests/RidTests.cs Lines 22 to 23 in 042a350 So, it's a known ambiguity with the parser. The parsing takes into account an optional qualifier that may exist at the end of the rid, which is also separated by a I'm not sure what the desired solution is. One option would be to make the parser aware of the architectures. We know the dashes before the @ericstj @ViktorHofer what do you think? |
|
I would expect this to be pre-loaded into the collection: runtime/src/libraries/Microsoft.NETCore.Platforms/src/GenerateRuntimeGraph.cs Lines 323 to 328 in 58d1ecb Later on, I'd expect this fast path to be hit: However it's not hit, since we hit the ambiguous case where linux-musl-x64 was parsed differently, so it's getting a different hashcode even though its string representation is identical.runtime/src/libraries/Microsoft.NETCore.Platforms/src/RID.cs Lines 190 to 194 in 58d1ecb One fix would be to use a comparer for this hashset that treated RIDs that were the same string representation as equal. There's really not a case where we'd define structured RIDs from the runtime groups with this ambiguity - it's only comping from string parsing in which case we want to ignore that ambiguity.
That's actually ignoring Qualifier. The same check works for Version because the version delimiter can only occur once. I think one option would be to make the Parse option have a flag to ignore Qualifiers and then plumb this through for AdditionalRuntimeIdentifier. The use of qualifiers was something we did early on ( |
I had come to that conclusion as well. @ericstj I had also suggested another option:
Any thoughts about that?
Shall I go ahead and implement this? |
|
I'd prefer to not further complicate the parsing algorithm with lookahead searches for known architectures. If anything I'd prefer we simplify it. My order of preference would be:
|
I assume the path doesn't get hit because the Because we know the additional rids won't have a qualifier, my preferred option would be: Otherwise we still have issue when an additional runtime identifier has |
@ayakael because source-build uses a non-portable rid, it won't encounter this parse issue. |
|
I'm good with that. I'd even consider deprecating the qualifier completely if we thought we could remove the I'd also be OK with doing both 1&2. It's not correct to allow two different RIDs with the same string representation but different meaning into the graph. Things will break later on when writing the graph in this case. |
|
Yes, I'd like to backport this to |
|
I think the most important CI, is the one for You can make the request on the PR. The maintainers will decide. |
Description
The dotnet/runtime build fails when running on Alpine in source-build mode with a portable configuration:
The problem is most visible with .NET 6, because dotnet/runtime is built in portable mode as part of the end-to-end source-build process.
With .NET 7 and .NET 8, source-build only builds dotnet/runtime in non-portable mode, so this problem doesn't appear there out of the box. It only appears when building dotnet/runtime by itself, in a configuration that's not used by source-build.
Reproduction Steps
Create an alpine container using a
DockerfileThen
git clone https://github.com/dotnet/runtimein the container and try and build runtime in source-build mode in the appropriate branch:main:
release/7.0 and release/6.0
Expected behavior
Build works
Actual behavior
Build fails
Regression?
No response
Known Workarounds
No response
Configuration
No response
Other information
No response
The text was updated successfully, but these errors were encountered: