Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Considerably speed up building the test tree on Windows #82080

Closed
wants to merge 1 commit into from
Closed

Considerably speed up building the test tree on Windows #82080

wants to merge 1 commit into from

Conversation

MichalStrehovsky
Copy link
Member

Building the JIT\Directed tree on my machine:

Before: 5 minutes 7 seconds.
After: 3 minutes 34 seconds.

Not kidding.

We run vswhere and vcvarsall. My bet is on vcvarsall - opening the visual studio tools command prompt always takes an eternity.

Cc @dotnet/ilc-contrib

@MichalStrehovsky
Considerably speed up building the test tree on Windows
f4b9091 
Building the JIT\Directed tree on my machine before: 5 minutes 7 seconds. After: 3 minutes 34 seconds.
@ghost
Copy link

ghost commented Feb 14, 2023

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

Building the JIT\Directed tree on my machine:

Before: 5 minutes 7 seconds.
After: 3 minutes 34 seconds.

Not kidding.

We run vswhere and vcvarsall. My bet is on vcvarsall - opening the visual studio tools command prompt always takes an eternity.

Cc @dotnet/ilc-contrib

Author: MichalStrehovsky
Assignees: -
Labels:

area-NativeAOT-coreclr
Milestone: -

@MichalStrehovsky
Copy link
Member Author

Hm, so one issue I just ran into with this is:

LINK : fatal error LNK1318: Unexpected PDB error; FILE_SYSTEM (3) 'C:\git\runtime\artifacts\tests\coreclr\windows.x64.R
elease\JIT\Performance\CodeQuality\Benchstones\BenchI\Puzzle\Puzzle\native\Puzzle.pdb' [C:\git\runtime\src\tests\JIT\Pe
rformance\CodeQuality\Benchstones\BenchI\Puzzle\Puzzle.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1201: error writing to program database 'C:\git\runtime\artifacts\tests\coreclr\windows.x64.Relea
se\JIT\Performance\CodeQuality\Benchstones\BenchI\Puzzle\Puzzle\native\Puzzle.pdb'; check for insufficient disk space,
invalid path, or insufficient privilege [C:\git\runtime\src\tests\JIT\Performance\CodeQuality\Benchstones\BenchI\Puzzle
\Puzzle.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\BenchI\Puzzle\Puzzle\native\link.rsp"" exited with code 1201. [C:\git\runtime\src\tests\JIT\Performance\C
odeQuality\Benchstones\BenchI\Puzzle\Puzzle.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1318: Unexpected PDB error; FILE_SYSTEM (3) 'C:\git\runtime\artifacts\tests\coreclr\windows.x64.R
elease\JIT\Performance\CodeQuality\Benchstones\BenchI\TreeInsert\TreeInsert\native\TreeInsert.pdb' [C:\git\runtime\src\
tests\JIT\Performance\CodeQuality\Benchstones\BenchI\TreeInsert\TreeInsert.csproj] [C:\git\runtime\src\tests\build.proj
]
LINK : fatal error LNK1201: error writing to program database 'C:\git\runtime\artifacts\tests\coreclr\windows.x64.Relea
se\JIT\Performance\CodeQuality\Benchstones\BenchI\TreeInsert\TreeInsert\native\TreeInsert.pdb'; check for insufficient
disk space, invalid path, or insufficient privilege [C:\git\runtime\src\tests\JIT\Performance\CodeQuality\Benchstones\B
enchI\TreeInsert\TreeInsert.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\BenchI\TreeInsert\TreeInsert\native\link.rsp"" exited with code 1201. [C:\git\runtime\src\tests\JIT\Perfo
rmance\CodeQuality\Benchstones\BenchI\TreeInsert\TreeInsert.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1201: error writing to program database 'C:\git\runtime\artifacts\tests\coreclr\windows.x64.Relea
se\JIT\Performance\CodeQuality\Benchstones\BenchI\QuickSort\QuickSort\native\QuickSort.pdb'; check for insufficient dis
k space, invalid path, or insufficient privilege [C:\git\runtime\src\tests\JIT\Performance\CodeQuality\Benchstones\Benc
hI\QuickSort\QuickSort.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\BenchI\QuickSort\QuickSort\native\link.rsp"" exited with code 1201. [C:\git\runtime\src\tests\JIT\Perform
ance\CodeQuality\Benchstones\BenchI\QuickSort\QuickSort.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1318: Unexpected PDB error; FILE_SYSTEM (3) 'C:\git\runtime\artifacts\tests\coreclr\windows.x64.R
elease\JIT\Performance\CodeQuality\Benchstones\BenchI\XposMatrix\XposMatrix\native\XposMatrix.pdb' [C:\git\runtime\src\
tests\JIT\Performance\CodeQuality\Benchstones\BenchI\XposMatrix\XposMatrix.csproj] [C:\git\runtime\src\tests\build.proj
]
LINK : fatal error LNK1201: error writing to program database 'C:\git\runtime\artifacts\tests\coreclr\windows.x64.Relea
se\JIT\Performance\CodeQuality\Benchstones\BenchI\XposMatrix\XposMatrix\native\XposMatrix.pdb'; check for insufficient
disk space, invalid path, or insufficient privilege [C:\git\runtime\src\tests\JIT\Performance\CodeQuality\Benchstones\B
enchI\XposMatrix\XposMatrix.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\BenchI\XposMatrix\XposMatrix\native\link.rsp"" exited with code 1201. [C:\git\runtime\src\tests\JIT\Perfo
rmance\CodeQuality\Benchstones\BenchI\XposMatrix\XposMatrix.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1201: error writing to program database 'C:\git\runtime\artifacts\tests\coreclr\windows.x64.Relea
se\JIT\Performance\CodeQuality\Benchstones\BenchI\TreeSort\TreeSort\native\TreeSort.pdb'; check for insufficient disk s
pace, invalid path, or insufficient privilege [C:\git\runtime\src\tests\JIT\Performance\CodeQuality\Benchstones\BenchI\
TreeSort\TreeSort.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\BenchI\TreeSort\TreeSort\native\link.rsp"" exited with code 1201. [C:\git\runtime\src\tests\JIT\Performan
ce\CodeQuality\Benchstones\BenchI\TreeSort\TreeSort.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1201: error writing to program database 'C:\git\runtime\artifacts\tests\coreclr\windows.x64.Relea
se\JIT\Performance\CodeQuality\Benchstones\MDBenchF\MDInProd\MDInProd\native\MDInProd.pdb'; check for insufficient disk
 space, invalid path, or insufficient privilege [C:\git\runtime\src\tests\JIT\Performance\CodeQuality\Benchstones\MDBen
chF\MDInProd\MDInProd.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\MDBenchF\MDInProd\MDInProd\native\link.rsp"" exited with code 1201. [C:\git\runtime\src\tests\JIT\Perform
ance\CodeQuality\Benchstones\MDBenchF\MDInProd\MDInProd.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1318: Unexpected PDB error; RPC (23) '(0x000006BE)' [C:\git\runtime\src\tests\JIT\Performance\Cod
eQuality\Benchstones\MDBenchI\MDLogicArray\MDLogicArray.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1318: Unexpected PDB error; RPC (23) '(0x000006BE)' [C:\git\runtime\src\tests\JIT\Performance\Cod
eQuality\Benchstones\MDBenchI\MDMulMatrix\MDMulMatrix.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1318: Unexpected PDB error; RPC (23) '(0x000006BE)' [C:\git\runtime\src\tests\JIT\Performance\Cod
eQuality\Benchstones\MDBenchI\MDGeneralArray\MDGeneralArray.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1318: Unexpected PDB error; RPC (23) '(0x000006BE)' [C:\git\runtime\src\tests\JIT\Performance\Cod
eQuality\Benchstones\MDBenchI\MDMidpoint\MDMidpoint.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1318: Unexpected PDB error; RPC (23) '(0x000006BA)' [C:\git\runtime\src\tests\JIT\Performance\Cod
eQuality\Benchstones\MDBenchF\MDSqMtx\MDSqMtx.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\MDBenchF\MDSqMtx\MDSqMtx\native\link.rsp"" exited with code 1318. [C:\git\runtime\src\tests\JIT\Performan
ce\CodeQuality\Benchstones\MDBenchF\MDSqMtx\MDSqMtx.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\MDBenchI\MDMulMatrix\MDMulMatrix\native\link.rsp"" exited with code 1318. [C:\git\runtime\src\tests\JIT\P
erformance\CodeQuality\Benchstones\MDBenchI\MDMulMatrix\MDMulMatrix.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\MDBenchI\MDGeneralArray\MDGeneralArray\native\link.rsp"" exited with code 1318. [C:\git\runtime\src\tests
\JIT\Performance\CodeQuality\Benchstones\MDBenchI\MDGeneralArray\MDGeneralArray.csproj] [C:\git\runtime\src\tests\build
.proj]
LINK : fatal error LNK1318: Unexpected PDB error; RPC (23) '(0x000006BA)' [C:\git\runtime\src\tests\JIT\Performance\Cod
eQuality\Benchstones\MDBenchF\MDRomber\MDRomber.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\MDBenchI\MDLogicArray\MDLogicArray\native\link.rsp"" exited with code 1318. [C:\git\runtime\src\tests\JIT
\Performance\CodeQuality\Benchstones\MDBenchI\MDLogicArray\MDLogicArray.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\MDBenchI\MDMidpoint\MDMidpoint\native\link.rsp"" exited with code 1318. [C:\git\runtime\src\tests\JIT\Per
formance\CodeQuality\Benchstones\MDBenchI\MDMidpoint\MDMidpoint.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\MDBenchF\MDRomber\MDRomber\native\link.rsp"" exited with code 1318. [C:\git\runtime\src\tests\JIT\Perform
ance\CodeQuality\Benchstones\MDBenchF\MDRomber\MDRomber.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1318: Unexpected PDB error; RPC (23) '(0x000006BA)' [C:\git\runtime\src\tests\JIT\Performance\Cod
eQuality\Benchstones\MDBenchF\MDLLoops\MDLLoops.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\MDBenchF\MDLLoops\MDLLoops\native\link.rsp"" exited with code 1318. [C:\git\runtime\src\tests\JIT\Perform
ance\CodeQuality\Benchstones\MDBenchF\MDLLoops\MDLLoops.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1318: Unexpected PDB error; RPC (23) '(0x000006BA)' [C:\git\runtime\src\tests\JIT\Performance\Cod
eQuality\Benchstones\MDBenchI\MDAddArray2\MDAddArray2.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\MDBenchI\MDAddArray2\MDAddArray2\native\link.rsp"" exited with code 1318. [C:\git\runtime\src\tests\JIT\P
erformance\CodeQuality\Benchstones\MDBenchI\MDAddArray2\MDAddArray2.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\src\tests\Common\dir.traversal.targets(25,5): error : (No message specified) [C:\git\runtime\src\tests\b
uild.proj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\src\tests\build.proj(496,5): error MSB3073: The command ""C:\git\runtime\\dotnet.cmd" msbuild C:\git\run
time\src\tests\build.proj /t:Build "/p:TargetArchitecture=x64" "/p:Configuration=Release" "/p:LibrariesConfiguration=Re
lease" "/p:TargetOS=windows" "/p:RuntimeOS=" "/p:RuntimeFlavor=coreclr" "/p:RuntimeVariant=" "/p:CLRTestBuildAllTargets
=" "/p:UseCodeFlowEnforcement=" "/p:__TestGroupToBuild=2" "/p:__SkipRestorePackages=1" /nodeReuse:false /maxcpucount /b
l:C:\git\runtime\artifacts\/log/Release/InnerManagedTestBuild.2.binlog "/p:DefaultBuildAllTarget=BuildNativeAot"" exite
d with code 1.

The fatal error LNK1318: Unexpected PDB error; RPC (23) is a mystery error that could be related to link.exe running out of memory. Indeed the logs have extra link.exe output where it says it's restarting as x64 because it ran out of memory.

The repo builds with x86_amd64 cross linker (#76516). The linker ILC would set up is the amd64 native linker.

I'm converting this to draft while I meditate on this. It's possible the compile time wins are also due to using the x86 linker. But also if there's stability issues, there's little point in doing this :(

@MichalStrehovsky MichalStrehovsky marked this pull request as draft February 15, 2023 06:18
@jkotas
Copy link
Member

jkotas commented Feb 15, 2023

Also, do instructions for running individual tests need updating for this change? https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/nativeaot.md#running-tests

@MichalStrehovsky
Copy link
Member Author

We can only do this if #76516 is fixed.

@MichalStrehovsky MichalStrehovsky deleted the fastbuild branch February 24, 2023 06:19
@ghost ghost locked as resolved and limited conversation to collaborators Mar 26, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees

@MichalStrehovsky MichalStrehovsky

Labels
area-NativeAOT-coreclr
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@MichalStrehovsky @jkotas