Move tests and test debuggees to repo root 'tests' directory #5554

max-charlamb · 2025-08-28T01:00:58Z

This pull request reorganizes and improves the project and build configuration, with a particular focus on test infrastructure and project structure. The changes streamline how test projects are referenced and built, clarify property settings for packaging and shipping, and update solution/project references to reflect a new directory structure for test assets.

Project and Build Configuration Improvements:

Added a new Build.proj traversal project that includes all main diagnostic tool projects and conditionally includes test projects, improving build orchestration and clarity.
Updated Directory.Build.props to set IsShipping and IsPackable to false by default, and suppress warnings about packing non-packable projects, ensuring that internal/test projects are not accidentally shipped or packed.
Added architecture-specific properties for test .NET install roots and registry roots, improving test environment configuration for different target architectures.

Test Infrastructure and Project Structure:

Added add_subdirectory(tests) to CMakeLists.txt to ensure that test projects are included in native builds.
Updated the Visual Studio solution file debuggees.sln to reference test debuggee projects from the new tests/Debuggees directory, and added several new test debuggee projects to the solution.
Corrected a path in THIRD-PARTY-NOTICES.TXT to point to the new location of the Xunit.Extensions code under tests/Microsoft.Diagnostics.TestHelpers, reflecting the directory reorganization.edited version of Move tests and test debuggees to repo root 'tests' directory #5469

…es with build in them. Flags as duplicate params

… (NuGet.config, Version.props, and a debuggee Directory.Build.props+targets)

eng/Build.props

eng/build.ps1

noahfalk

What is the rationale for moving stuff in \src\tests -> \tests? For example dotnet\runtime or dotnet\aspnetcore both put their tests under the src directory. It seems like this creates a lot of churn and makes our repo different than others.

I'm not necessarily opposed but I do want to understand before we churn 100s of files in the repo versioning history. I just marked request changes to make sure it doesn't get accidentally checked in before we talked about it.

steveisok

I think we're good now.

src/tests/Directory.Build.props

src/tests/DbgShim.UnitTests/ConfigFiles/Windows/Debugger.Tests.Config.txt

noahfalk · 2025-09-05T23:25:56Z

src/tests/DbgShim.UnitTests/ConfigFiles/Unix/Debugger.Tests.Config.txt

  <Import ConfigFile="Debugger.Tests.Common.txt" />
-  <DotNetRoot>$(RepoRootDir)/.dotnet-test</DotNetRoot>
-  <Import ConfigFile="$(DotNetRoot)/Debugger.Tests.Versions.txt" />
+  <DotNetRoot Condition="'$(TestDotNetRoot)' != ''">$(TestDotNetRoot)</DotNetRoot>


I assume the design intent is that TestDotNetRoot won't be defined when running Helix? Its not yet clear to me why that will be the case, but I'm hoping things will make sense once I've got a better understanding how the Helix run is executed.

Yeah, that's the only thing that can't be relative pathed. When the sdk is sent along, the DOTNET_ROOT env variable is also set. We can just read from the env variable in that case.

From what I see in the logic, DotNetRoot gets set to the env var DOTNET_ROOT before we start parsing the config file. TestDotNetRoot gets set from Debugger.Tests.Common.txt which was generated in the build and points to the '.dotnet-test' folder. If the files were then copied to helix and run with DOTNET_ROOT set it looks like the final result would be ".dotnet-test" which isn't the behavior we want right?

I was expecting this would be something like "<DotNetRoot Condition='$(DotNetRoot)'==''>$(DefaultDotNetRoot) where DefaultDotNetRoot = $(RepoRoot).dotnet-test[\x86]

I'll revert this change as it doesn't really apply until #5560

noahfalk · 2025-09-26T22:34:35Z

debuggees.sln

 VisualStudioVersion = 17.12.35309.182
 MinimumVisualStudioVersion = 10.0.40219.1
-Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "TestDebuggee", "src\SOS\lldbplugin.tests\TestDebuggee\TestDebuggee.csproj", "{6C43BE85-F8C3-4D76-8050-F25CE953A7FD}"
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "TestDebuggee", "src\tests\Debuggees\TestDebuggee\TestDebuggee.csproj", "{6C43BE85-F8C3-4D76-8050-F25CE953A7FD}"


As I understand the reason we are moving all the debuggees around is to facilitate carving up the bin directory into smaller per-test artifacts that can be sent to Helix. I'd propose we start with a simpler model and then observe what performance we are getting before doing further optimization:

Package up the entire artifacts/bin into a single bundle (I'm seeing 280MB compressed)

We can still choose to send that bundle to multiple machines and run a single test project on each if we want (at least I assume we can do that).

That should eliminate the need to move files around, track which projects depend on which assets, or create any complicated bundling logic.

I get what you're after. I'll tweak this to have the artifacts land where they normally would and then we can address in #5560

noahfalk · 2025-09-26T23:50:55Z

src/tests/DbgShim.UnitTests/ConfigFiles/Unix/Debugger.Tests.Config.txt

-  <DotNetRoot>$(RepoRootDir)/.dotnet-test</DotNetRoot>
-  <Import ConfigFile="$(DotNetRoot)/Debugger.Tests.Versions.txt" />
+  <DotNetRoot Condition="'$(TestDotNetRoot)' != ''">$(TestDotNetRoot)</DotNetRoot>
+  <Import ConfigFile="$(TestVersionConfigDir)/Debugger.Tests.Versions.txt" />


It feels weird that we've got some tests looking for this version file in one place (.dotnet-test) and other tests looking for it in artifacts\bin. Even if it is hard-coded for now, can we get them all pointing to the same place and remove the versions.config.txt being produced in .dotnet-test?

It would be easier to revert this change as it doesn't really apply until #5560.

That works too.

noahfalk · 2025-09-26T23:51:51Z

src/tests/DbgShim.UnitTests/ConfigFiles/Unix/Debugger.Tests.Config.txt

  <Import ConfigFile="Debugger.Tests.Common.txt" />
-  <DotNetRoot>$(RepoRootDir)/.dotnet-test</DotNetRoot>
-  <Import ConfigFile="$(DotNetRoot)/Debugger.Tests.Versions.txt" />
+  <DotNetRoot Condition="'$(TestDotNetRoot)' != ''">$(TestDotNetRoot)</DotNetRoot>


From what I see in the logic, DotNetRoot gets set to the env var DOTNET_ROOT before we start parsing the config file. TestDotNetRoot gets set from Debugger.Tests.Common.txt which was generated in the build and points to the '.dotnet-test' folder. If the files were then copied to helix and run with DOTNET_ROOT set it looks like the final result would be ".dotnet-test" which isn't the behavior we want right?

I was expecting this would be something like "<DotNetRoot Condition='$(DotNetRoot)'==''>$(DefaultDotNetRoot) where DefaultDotNetRoot = $(RepoRoot).dotnet-test[\x86]

noahfalk · 2025-09-27T03:32:23Z

src/tests/Directory.Build.targets

+      <FileWrites Include="$(DebuggerVersionConfigFileName)" />
+    </ItemGroup>
+  </Target>
+</Project>


I wanted to comment on the *.UnitTest renames but GH won't let me add a comment there.

I'd suggest if we rename these projects we should use ".Tests" instead of ".UnitTests". Many of these tests are more accurately described as functional or integration tests rather than unit tests.

Sounds good to me.

Mike McLaughlin and others added 14 commits May 29, 2025 09:45

Move tests and test debuggees to repo root 'tests' directory

8186c59

Remove debuggee from DbgShim.UnitTest project

ad7cdcd

Everything under the tests directory is not packable

1f9a870

Fix packing in official build

93a5efa

Fix single-file build step

a57bf97

Merge remote-tracking branch 'upstream/main' into organize_tests

b8315c8

Merge remote-tracking branch 'upstream/main' into organize_tests

cd2e7d8

Split out build tools and tests from running them

aebac9c

Use -withtests instead because powershell doesn't seem to like prefix…

7c06f2d

…es with build in them. Flags as duplicate params

WIP - DbgShim.UnitTests can run outside of the repo with a few tweaks…

b118af2

… (NuGet.config, Version.props, and a debuggee Directory.Build.props+targets)

Merge remote-tracking branch 'upstream/main' into organize_tests

2a8905b

fix paths

6eecb79

Merge branch 'main' into organize_tests2

37097ef

modify build.proj

878a01a

jkoritzinsky reviewed Aug 28, 2025

View reviewed changes

eng/Build.props Outdated Show resolved Hide resolved

max-charlamb added 4 commits August 28, 2025 13:56

fix source-build

56fd8e8

support helix in build.ps1

2ed0c53

remove references to helix

1c87d1c

remove helix references

418570a

max-charlamb marked this pull request as ready for review August 28, 2025 21:46

max-charlamb requested a review from a team as a code owner August 28, 2025 21:46

max-charlamb changed the title ~~Prepare tests to run on helix~~ Move tests and test debuggees to repo root 'tests' directory Aug 28, 2025

This was referenced Aug 29, 2025

Support running test out of proc #5560

Draft

[cDAC] Run runtime-diagnostics pipeline on more platforms dotnet/runtime#119212

Open

steveisok previously approved these changes Aug 29, 2025

View reviewed changes

modify cli arguments

5dc3574

max-charlamb dismissed steveisok’s stale review via 5dc3574 August 29, 2025 18:48

max-charlamb commented Aug 29, 2025

View reviewed changes

eng/build.ps1 Show resolved Hide resolved

max-charlamb added 2 commits August 29, 2025 14:59

Update eng/build.ps1

12da84a

fix pipeline

4d0b29f

undo change

7a4f23e

steveisok self-requested a review August 29, 2025 22:28

steveisok previously approved these changes Aug 29, 2025

View reviewed changes

noahfalk requested changes Aug 29, 2025

View reviewed changes

Move tests into src/tests and adjust any pathing weirdness

73ff521

steveisok dismissed their stale review via 73ff521 September 4, 2025 18:53

steveisok added 3 commits September 4, 2025 15:03

Add binaries and other ignored files that were in tree back

a6e0aaf

Merge remote-tracking branch 'upstream/main' into organize_tests2

f615832

Correct lldbplugin.tests location

8b10fe4

steveisok self-requested a review September 4, 2025 23:42

steveisok previously approved these changes Sep 4, 2025

View reviewed changes

noahfalk reviewed Sep 5, 2025

View reviewed changes

src/tests/Directory.Build.props Show resolved Hide resolved

src/tests/DbgShim.UnitTests/ConfigFiles/Windows/Debugger.Tests.Config.txt Outdated Show resolved Hide resolved

src/tests/DbgShim.UnitTests/ConfigFiles/Windows/Debugger.Tests.Config.txt Outdated Show resolved Hide resolved

noahfalk reviewed Sep 5, 2025

View reviewed changes

noahfalk reviewed Sep 27, 2025

View reviewed changes

Revert changes intended for running tests standalone

ac4805a

steveisok dismissed their stale review via ac4805a September 29, 2025 20:44

Rename .UnitTests to .Tests

cd7353b

Move tests and test debuggees to repo root 'tests' directory #5554

Are you sure you want to change the base?

Move tests and test debuggees to repo root 'tests' directory #5554

Uh oh!

Conversation

max-charlamb commented Aug 28, 2025 • edited by steveisok Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

noahfalk left a comment

Choose a reason for hiding this comment

Uh oh!

steveisok left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

max-charlamb commented Aug 28, 2025 •

edited by steveisok

Loading