Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot test on Arm64 : Could not copy crossgen2_publish apphost #107457

Closed
a74nh opened this issue Sep 6, 2024 · 6 comments
Closed

Cannot test on Arm64 : Could not copy crossgen2_publish apphost #107457

a74nh opened this issue Sep 6, 2024 · 6 comments
Labels
area-Infrastructure-coreclr

Comments

@a74nh
Copy link
Contributor

a74nh commented Sep 6, 2024 •
edited
Loading

With latest HEAD. On Arm64 Linux.

rm -fr artifacts/
./build.sh -rc' checked -lc release -s clr+libs
./src/tests/build.sh generatelayoutonly Checked

Errors with:

/home/alahay01/dotnet/runtime_base/.dotnet/sdk/9.0.100-preview.7.24407.12/Microsoft.Common.CurrentVersion.targets(5327,5): error MSB3030: Could not copy the file "/home/alahay01/dotnet/runtime_base/artifacts/obj/coreclr/crossgen2_publish/linux.arm64.Checked/apphost" because it was not found. [/home/alahay01/dotnet/runtime_base/src/coreclr/tools/aot/crossgen2/crossgen2_publish.csproj]

Related to #106965 ??

This stops me from building and running the testsuite.
@dotnet-policy-service dotnet-policy-service bot added the untriaged New issue has not been triaged by the area owner label Sep 6, 2024
@dotnet-policy-service Dotnet Policy Service
Copy link
Contributor

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

@a74nh
Copy link
Contributor Author

a74nh commented Sep 6, 2024

Full generatelayoutonly log:

❯ ./src/tests/build.sh generatelayoutonly Checked
__OutputRid: linux-arm64
Building Tests...
__TargetOS: linux
__TargetArch: arm64
__BuildType: Checked
__TestIntermediatesDir: /home/alahay01/dotnet/runtime_base/artifacts/tests/coreclr/obj/linux.arm64.Checked
__NativeTestIntermediatesDir: /home/alahay01/dotnet/runtime_base/artifacts/tests/coreclr/obj/linux.arm64.Checked/Native
__ManagedTestIntermediatesDir: /home/alahay01/dotnet/runtime_base/artifacts/tests/coreclr/obj/linux.arm64.Checked/Managed
Creating TestBinDir: /home/alahay01/dotnet/runtime_base/artifacts/tests/coreclr/linux.arm64.Checked
Creating LogsDir: /home/alahay01/dotnet/runtime_base/artifacts/log
Creating MsbuildDebugLogsDir: /home/alahay01/dotnet/runtime_base/artifacts/log/MsbuildDebugLogs
Building tests via "/home/alahay01/dotnet/runtime_base/eng/common/msbuild.sh"  --warnAsError false /home/alahay01/dotnet/runtime_base/src/tests/build.proj /t:TestBuild /p:TargetArchitecture=arm64 /p:Configuration=Checked /p:TargetOS=linux /nodeReuse:false    /maxcpucount "/flp:Verbosity=normal;LogFile=/home/alahay01/dotnet/runtime_base/artifacts/log/TestBuild.linux.arm64.Checked.log" "/flp1:WarningsOnly;LogFile=/home/alahay01/dotnet/runtime_base/artifacts/log/TestBuild.linux.arm64.Checked.wrn" "/flp2:ErrorsOnly;LogFile=/home/alahay01/dotnet/runtime_base/artifacts/log/TestBuild.linux.arm64.Checked.err" "/bl:/home/alahay01/dotnet/runtime_base/artifacts/log/TestBuild.linux.arm64.Checked.binlog" /p:NUMBER_OF_PROCESSORS=80

  [11:52:08.96] Restoring all packages...

    Determining projects to restore...
    Restored /home/alahay01/dotnet/runtime_base/src/tests/Common/test_dependencies_fs/test_dependencies.fsproj (in 1.2 sec).

    Determining projects to restore...
    Restored /home/alahay01/dotnet/runtime_base/src/tests/Common/test_dependencies/test_dependencies.csproj (in 2.61 sec).

    Determining projects to restore...
    Restored /home/alahay01/dotnet/runtime_base/src/tests/Common/CoreCLRTestLibrary/CoreCLRTestLibrary.csproj (in 2.53 sec).

    Determining projects to restore...
    Restored /home/alahay01/dotnet/runtime_base/src/tests/Common/GenerateHWIntrinsicTests/GenerateHWIntrinsicTests_Arm.csproj (in 220 ms).

    Determining projects to restore...
    Restored /home/alahay01/dotnet/runtime_base/src/tests/Common/GenerateHWIntrinsicTests/GenerateHWIntrinsicTests_General.csproj (in 237 ms).

    Determining projects to restore...
    Restored /home/alahay01/dotnet/runtime_base/src/tests/Common/GenerateHWIntrinsicTests/GenerateHWIntrinsicTests_X86.csproj (in 215 ms).

    Determining projects to restore...
    Restored /home/alahay01/dotnet/runtime_base/src/tests/Common/XUnitLogChecker/XUnitLogChecker.csproj (in 1.77 sec).

    Determining projects to restore...
    Restored /home/alahay01/dotnet/runtime_base/src/tests/Common/XUnitWrapperGenerator/XUnitWrapperGenerator.csproj (in 342 ms).

    Determining projects to restore...
    Restored /home/alahay01/dotnet/runtime_base/src/tests/Common/XUnitWrapperLibrary/XUnitWrapperLibrary.csproj (in 218 ms).

    Determining projects to restore...
    Restored /home/alahay01/dotnet/runtime_base/src/tests/Common/XHarnessRunnerLibrary/XHarnessRunnerLibrary.csproj (in 2.15 sec).
    1 of 2 projects are up-to-date for restore.

    Determining projects to restore...
    Restored /home/alahay01/dotnet/runtime_base/src/tests/Common/external/external.csproj (in 2.44 sec).

    Determining projects to restore...
    Restored /home/alahay01/dotnet/runtime_base/src/tests/Common/ilasm/ilasm.ilproj (in 2.28 sec).
  [11:53:02.30] Restoring all packages...Done.
  Building Targeting Pack
  external ->
  Determining projects to restore...
  Restored /home/alahay01/dotnet/runtime_base/src/tools/illink/src/linker/ref/Mono.Linker.csproj (in 430 ms).
  Restored /home/alahay01/dotnet/runtime_base/src/coreclr/tools/aot/ILCompiler.TypeSystem/ILCompiler.TypeSystem.csproj (in 430 ms).
  Restored /home/alahay01/dotnet/runtime_base/src/coreclr/tools/aot/ILCompiler.ReadyToRun/ILCompiler.ReadyToRun.csproj (in 431 ms).
  Restored /home/alahay01/dotnet/runtime_base/src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/Microsoft.Interop.SourceGeneration.csproj (in 460 ms).
  Restored /home/alahay01/dotnet/runtime_base/src/coreclr/tools/aot/ILCompiler.Diagnostics/ILCompiler.Diagnostics.csproj (in 430 ms).
  Restored /home/alahay01/dotnet/runtime_base/src/coreclr/tools/aot/ILCompiler.DependencyAnalysisFramework/ILCompiler.DependencyAnalysisFramework.csproj (in 430 ms).
  Restored /home/alahay01/dotnet/runtime_base/src/tools/illink/src/ILLink.CodeFix/ILLink.CodeFixProvider.csproj (in 456 ms).
  Restored /home/alahay01/dotnet/runtime_base/src/tools/illink/src/linker/Mono.Linker.csproj (in 443 ms).
  Restored /home/alahay01/dotnet/runtime_base/src/coreclr/tools/aot/crossgen2/crossgen2_publish.csproj (in 430 ms).
  Restored /home/alahay01/dotnet/runtime_base/src/libraries/System.Runtime.InteropServices/gen/DownlevelLibraryImportGenerator/DownlevelLibraryImportGenerator.csproj (in 450 ms).
  Restored /home/alahay01/dotnet/runtime_base/src/tools/illink/src/ILLink.RoslynAnalyzer/ILLink.RoslynAnalyzer.csproj (in 450 ms).
  Restored /home/alahay01/dotnet/runtime_base/src/tools/illink/src/ILLink.Tasks/ILLink.Tasks.csproj (in 489 ms).
  Restored /home/alahay01/dotnet/runtime_base/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Crossgen2.sfxproj (in 1.99 sec).
  ILLink.RoslynAnalyzer -> /home/alahay01/dotnet/runtime_base/artifacts/bin/ILLink.RoslynAnalyzer/netstandard2.0/ILLink.RoslynAnalyzer.dll
  ILLink.CodeFixProvider -> /home/alahay01/dotnet/runtime_base/artifacts/bin/ILLink.CodeFixProvider/netstandard2.0/ILLink.CodeFixProvider.dll
  Mono.Linker -> /home/alahay01/dotnet/runtime_base/artifacts/bin/Mono.Linker/ref/net9.0/illink.dll
  Mono.Linker -> /home/alahay01/dotnet/runtime_base/artifacts/bin/Mono.Linker/net9.0/linux-arm64/illink.dll
  ILLink.Tasks -> /home/alahay01/dotnet/runtime_base/artifacts/bin/ILLink.Tasks/net9.0/linux-arm64/ILLink.Tasks.dll
  Mono.Linker -> /home/alahay01/dotnet/runtime_base/artifacts/bin/Mono.Linker/net9.0/illink.dll
  ILLink.Tasks -> /home/alahay01/dotnet/runtime_base/artifacts/bin/ILLink.Tasks/net9.0/ILLink.Tasks.dll
  ILCompiler.DependencyAnalysisFramework -> /home/alahay01/dotnet/runtime_base/artifacts/bin/ILCompiler.DependencyAnalysisFramework/arm64/Checked/ILCompiler.DependencyAnalysisFramework.dll
  ILCompiler.TypeSystem -> /home/alahay01/dotnet/runtime_base/artifacts/bin/ILCompiler.TypeSystem/arm64/Checked/ILCompiler.TypeSystem.dll
  ILCompiler.Diagnostics -> /home/alahay01/dotnet/runtime_base/artifacts/bin/ILCompiler.Diagnostics/arm64/Checked/ILCompiler.Diagnostics.dll
  ILCompiler.ReadyToRun -> /home/alahay01/dotnet/runtime_base/artifacts/bin/ILCompiler.ReadyToRun/arm64/Checked/ILCompiler.ReadyToRun.dll
/home/alahay01/dotnet/runtime_base/.dotnet/sdk/9.0.100-preview.7.24407.12/Microsoft.Common.CurrentVersion.targets(5327,5): error MSB3030: Could not copy the file "/home/alahay01/dotnet/runtime_base/artifacts/obj/coreclr/crossgen2_publish/linux.arm64.Checked/apphost" because it was not found. [/home/alahay01/dotnet/runtime_base/src/coreclr/tools/aot/crossgen2/crossgen2_publish.csproj]
  Determining projects to restore...
  All projects are up-to-date for restore.

Build FAILED.

/home/alahay01/dotnet/runtime_base/.dotnet/sdk/9.0.100-preview.7.24407.12/Microsoft.Common.CurrentVersion.targets(5327,5): error MSB3030: Could not copy the file "/home/alahay01/dotnet/runtime_base/artifacts/obj/coreclr/crossgen2_publish/linux.arm64.Checked/apphost" because it was not found. [/home/alahay01/dotnet/runtime_base/src/coreclr/tools/aot/crossgen2/crossgen2_publish.csproj]
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:01:16.16
Build failed with exit code 1. Check errors above.
Failed to build tests. See the build logs:
    /home/alahay01/dotnet/runtime_base/artifacts/log/TestBuild.linux.arm64.Checked.log
    /home/alahay01/dotnet/runtime_base/artifacts/log/TestBuild.linux.arm64.Checked.wrn
    /home/alahay01/dotnet/runtime_base/artifacts/log/TestBuild.linux.arm64.Checked.err
    /home/alahay01/dotnet/runtime_base/artifacts/log/TestBuild.linux.arm64.Checked.binlog

@am11
Copy link
Member

am11 commented Sep 6, 2024

./build.sh -rc' checked -lc release -s clr+libs

-lc should be -c.

Otherwise, it implies -hc Debug, i.e. host, libs and clr are in different configs which is not the intention, right?

@a74nh
Copy link
Contributor Author

a74nh commented Sep 6, 2024

./build.sh -rc' checked -lc release -s clr+libs

-lc should be -c.

Otherwise, it implies -hc Debug, i.e. host, libs and clr are in different configs which is not the intention, right?

That seems to fix it for me.

I guess what I should really be using is:
./build.sh -rc checked -lc release -hc checked-s clr+libs

My question then is has something changed to why this used to work?

@am11
Copy link
Member

am11 commented Sep 6, 2024

My question then is has something changed to why this used to work?

We were not requiring apphost before, now it is required by some bogus copy target (which copies apphost just so it stays unused.. long story short; it's another infra bug which needs fixing).

The old and current behaviors are not necessarily wrong. The core of the problem is that build.sh and src/tests/build.sh have different defaults when it comes to configuration. We should streamline them, make an announcement for folks to update their local workflows (which only takes few seconds) and document which combinations are supported. On community-supported platforms, options are further constrained due to lack of official SDK.

e.g. the logical counterpart of ./build.sh -rc Checked -c Debug should be src/tests/build.sh Debug -p:RuntimeConfiguration=Checked, but it is src/tests/build.sh Checked -p:LibrariesConfiguration=Debug. The default of libraires configuration (or any configuration: runtime, tools, host, libraries) is Debug for the top-level build script, but test build's default configuration for libraries is Release. So we had to do extra mapping to streamline things in test infra.

In general, it's a good idea to use one of the workflows exercised by CI (200+ legs) and those are also documented. Those workflows are continuously tested.

@a74nh
Copy link
Contributor Author

a74nh commented Sep 6, 2024

Thanks for the details. Closing this as I just need to update my workflow.

@a74nh a74nh closed this as completed Sep 6, 2024
@dotnet-policy-service dotnet-policy-service bot removed the untriaged New issue has not been triaged by the area owner label Sep 6, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Oct 7, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
area-Infrastructure-coreclr
Projects
Runtime Infra
Archived in project
Milestone
No milestone
Development

No branches or pull requests
2 participants
@am11 @a74nh