Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Opt Into Perf Logging #6274

Merged
merged 35 commits into from
Apr 15, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
24c3e42
Add DOTNET_PERFLOG_DIR environment variable when we call the build sc…
benvillalobos Mar 16, 2021
5c4781d
Publish perflogs for pipeline builds
benvillalobos Mar 16, 2021
26d8deb
Check if env var is null or empty, try creating the directory if it d…
benvillalobos Mar 16, 2021
ded3e03
Modify unit test to account for performance logger creating the direc…
benvillalobos Mar 16, 2021
fda5165
Strip away any quotes in the env var
benvillalobos Mar 19, 2021
7c8f351
Opt into perf logging during bootstrap builds
benvillalobos Mar 19, 2021
6babc3f
Revert "Add DOTNET_PERFLOG_DIR environment variable when we call the …
benvillalobos Mar 19, 2021
6ac7122
Revert "Publish perflogs for pipeline builds"
benvillalobos Mar 19, 2021
b7cbda6
Add publish step for perf logs in CI builds
benvillalobos Mar 19, 2021
2c96e62
Revert "Add publish step for perf logs in CI builds"
benvillalobos Mar 19, 2021
bb577df
Rename output directory to PerformanceLogs
benvillalobos Mar 19, 2021
f5dbbfe
Have the only non-bootstrap build run before all bootstrap builds
benvillalobos Mar 19, 2021
002145c
Dont pass a null string into TrimAndStripAnyQuotes
benvillalobos Mar 24, 2021
202e0fd
DEBUG: Add console writelines on perflogeventlistener creation
benvillalobos Mar 24, 2021
cde91c9
Add performance logging to cibuild_bootstrapped_msbuild shell script …
benvillalobos Mar 25, 2021
85c68db
Don't strip quotes from env var.
benvillalobos Mar 25, 2021
afb9c2b
Allow 5 retries for uploading perf logs.
benvillalobos Mar 25, 2021
83b808a
DEBUG: Add powershell script to output details on the process holding…
benvillalobos Mar 31, 2021
4b091df
DEBUG: Fix path in powershell script
benvillalobos Mar 31, 2021
84a1905
DEBUG: Powershell task has a name. Kills msbuild and vbcscompiler pro…
benvillalobos Apr 1, 2021
fc90d2a
Debug: Powershell task only kills processes
benvillalobos Apr 1, 2021
3280482
Revert "DEBUG: Add console writelines on perflogeventlistener creation"
benvillalobos Apr 2, 2021
eae3885
Revert "Allow 5 retries for uploading perf logs."
benvillalobos Apr 2, 2021
41f0fc6
Revert "Have the only non-bootstrap build run before all bootstrap bu…
benvillalobos Apr 2, 2021
2df6127
Run powershell script for correct windows build tasks
benvillalobos Apr 2, 2021
2fce6e2
Try naming powershell task for clarity
benvillalobos Apr 2, 2021
57987d3
Capitalize the displayname of the powershell task
benvillalobos Apr 2, 2021
f1230ac
DEBUG: Print out command line for lingering processes to find the log…
benvillalobos Apr 5, 2021
1cc8885
Fix indents
benvillalobos Apr 5, 2021
86be063
Run dotnet build-server shutdown instead of taskkill
benvillalobos Apr 5, 2021
1eed79b
Set environment variable to disable node reuse
benvillalobos Apr 5, 2021
6a4a460
Disable nodereuse for test that spawns lingering msbuild.exe
benvillalobos Apr 12, 2021
5f1fb73
Remove disablenodereuse env variable from cibuild script
benvillalobos Apr 12, 2021
424e50a
Prevent test assemblies from logging performance info by clearing the…
benvillalobos Apr 13, 2021
32900ee
Revert "Disable nodereuse for test that spawns lingering msbuild.exe"
benvillalobos Apr 13, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions eng/cibuild_bootstrapped_msbuild.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ $RepoRoot = [System.IO.Path]::GetFullPath($RepoRoot).TrimEnd($([System.IO.Path]:
$ArtifactsDir = Join-Path $RepoRoot "artifacts"
$Stage1Dir = Join-Path $RepoRoot "stage1"
$Stage1BinDir = Join-Path $Stage1Dir "bin"
$PerfLogDir = Join-Path $ArtifactsDir "log\$Configuration\PerformanceLogs"

if ($msbuildEngine -eq '')
{
Expand Down Expand Up @@ -123,6 +124,9 @@ try {
# Ensure that debug bits fail fast, rather than hanging waiting for a debugger attach.
$env:MSBUILDDONOTLAUNCHDEBUGGER="true"

# Opt into performance logging. https://github.com/dotnet/msbuild/issues/5900
$env:DOTNET_PERFLOG_DIR=$PerfLogDir
benvillalobos marked this conversation as resolved.
Show resolved Hide resolved

# When using bootstrapped MSBuild:
# - Turn off node reuse (so that bootstrapped MSBuild processes don't stay running and lock files)
# - Do run tests
Expand Down
4 changes: 4 additions & 0 deletions eng/cibuild_bootstrapped_msbuild.sh
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ done
RepoRoot="$ScriptRoot/.."
ArtifactsDir="$RepoRoot/artifacts"
Stage1Dir="$RepoRoot/stage1"
PerfLogDir="$ArtifactsDir/log/$configuration/PerformanceLogs"

. "$ScriptRoot/common/tools.sh"
InitializeDotNetCli true
Expand Down Expand Up @@ -86,6 +87,9 @@ mv $ArtifactsDir $Stage1Dir
# Ensure that debug bits fail fast, rather than hanging waiting for a debugger attach.
export MSBUILDDONOTLAUNCHDEBUGGER=true

# Opt into performance logging.
export DOTNET_PERFLOG_DIR=$PerfLogDir

# Prior to 3.0, the Csc task uses this environment variable to decide whether to run
# a CLI host or directly execute the compiler.
export DOTNET_HOST_PATH="$_InitializeDotNetCli/dotnet"
Expand Down
4 changes: 2 additions & 2 deletions src/MSBuild.UnitTests/PerfLog_Tests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ public void TestPerfLogEnabledProducedLogFile()
}

[Fact]
public void TestPerfLogDirectoryDoesNotExist()
public void TestPerfLogDirectoryGetsCreated()
benvillalobos marked this conversation as resolved.
Show resolved Hide resolved
{
using (TestEnvironment testEnv = TestEnvironment.Create(_output))
{
Expand All @@ -101,7 +101,7 @@ public void TestPerfLogDirectoryDoesNotExist()
RunnerUtilities.ExecMSBuild(msbuildParameters, out bool successfulExit);
successfulExit.ShouldBeTrue();

Directory.Exists(perfLogPath).ShouldBeFalse();
Directory.Exists(perfLogPath).ShouldBeTrue();
}
}
}
Expand Down
4 changes: 3 additions & 1 deletion src/MSBuild/PerformanceLogEventListener.cs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
using System.IO;
using System.Text;
using Microsoft.Build.Eventing;
using Microsoft.Build.Shared;

namespace Microsoft.Build.CommandLine
{
Expand Down Expand Up @@ -51,7 +52,8 @@ internal static PerformanceLogEventListener Create()

// Check to see if we should enable the event listener.
string logDirectory = Environment.GetEnvironmentVariable(PerfLogDirEnvVar);
if (Directory.Exists(logDirectory))

if (!string.IsNullOrEmpty(logDirectory) && Directory.CreateDirectory(logDirectory).Exists)
{
eventListener = new PerformanceLogEventListener();
eventListener.Initialize(logDirectory);
Expand Down
4 changes: 4 additions & 0 deletions src/Shared/UnitTests/TestAssemblyInfo.cs
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@ public MSBuildTestAssemblyFixture()
// (VerifySubToolsetVersionSetByConstructorOverridable), as the environment variable would take precedence.
_testEnvironment.SetEnvironmentVariable("VisualStudioVersion", string.Empty);

// Prevent test assemblies from logging any performance info.
// https://github.com/dotnet/msbuild/pull/6274
_testEnvironment.SetEnvironmentVariable("DOTNET_PERFLOG_DIR", string.Empty);

SetDotnetHostPath(_testEnvironment);

// Use a project-specific temporary path
Expand Down