This repository has been archived by the owner on Jan 23, 2023. It is now read-only.
BenchmarkDotNet can benchmark local CoreFX builds using CoreRun #31921
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
adamsitnik
added
the
documentation
|
|
||
| We recommend using [BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet) as it allows specifying custom SDK paths and measuring performance not just in-proc but also out-of-proc as a dedicated executable. | ||
|
|
||
| ``` | ||
| <ItemGroup> | ||
| <PackageReference Include="BenchmarkDotNet" Version="0.10.13" /> | ||
| <PackageReference Include="BenchmarkDotNet" Version="0.11.1" /> | ||
| </ItemGroup> | ||
| ``` | ||
|
|
||
| ## Defining your benchmark | ||
|
|
||
| See [BenchmarkDotNet](http://benchmarkdotnet.org/Guides/GettingStarted.htm) documentation -- minimally you need to adorn a public method with the `[Benchmark]` attribute but there are many other ways to customize what is done such as using parameter sets or setup/cleanup methods. Of course, you'll want to bracket just the relevant code in your benchmark, ensure there are sufficient iterations that you minimise noise, as well as leaving the machine otherwise idle while you measure. |
There was a problem hiding this comment.
This link should be updated: https://benchmarkdotnet.org/articles/guides/getting-started.html
I added redirects for all "old-style" doc pages, so people will see this page anyway, but I still want to update links wherever possible.
danmoseley
approved these changes
Aug 24, 2018
|
|
||
| **Important:** BenchmarkDotNet will generate the right `.csproj` file for the self-contained app. It's going to reference the `.csproj` file of the project which defines benchmarks. It's going to work even if your project is not self-contained app targeting local CoreFX build. So you can just create a new solution with console app in Visual Studio, install BenchmarkDotNet and it's going to do the right thing for you. | ||
| dotnet run -c Release -f netcoreapp2.1 -- -f *MyBenchmarkName* --coreRun "C:\Projects\corefx\bin\testhost\netcoreapp-Windows_NT-Release-x64\shared\Microsoft.NETCore.App\9.9.9\CoreRun.exe" | ||
|
|
||
| **Hint:** If you are curious to know what BDN does internally you just need to apply `[KeepBenchmarkFiles]` attribute to your class or set `KeepBenchmarkFiles = true` in your config file. After runing the benchmarks you can find the auto-generated files in `%pathToBenchmarkApp\bin\Release\$TFM\` folder. |
| Example: | ||
| 2. Create a new .NET Core console app using your favorite IDE | ||
| 3. Install BenchmarkDotNet (0.11.1+) | ||
| 4. Define the benchmarks |
| using BenchmarkDotNet.Loggers; | ||
| using BenchmarkDotNet.Toolchains.CsProj; | ||
| using BenchmarkDotNet.Toolchains.DotNetCli; | ||
| 9. Run the benchmarks using `--coreRun` from the fist step. Save the results in a dedicated folder. |
adamsitnik
commented
Aug 24, 2018
|
|
||
| **Hint:** If you would like to compare the performance of different CoreFX versions, you just need to define multiple jobs, each using it's own toolchain. | ||
| 1. Before you start benchmarking the code you need to build entire CoreFX in Release which is going to generate the right CoreRun bits for you: |
There was a problem hiding this comment.
@danmosemsft the missing #1 is here
picenka21
pushed a commit
to picenka21/runtime
that referenced
this pull request
Feb 18, 2022
…et/corefx#31921) * 0.11.1 knows how to use CoreRun to run the local CoreFX benchmarks! * fix typos and old link Commit migrated from dotnet/corefx@3475a8d
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
BDN can now run benchmarks using CoreRun and all you need to do is to provide the path to CoreRun using console argument.
It simplifies everything and makes it very easy to benchmark local CoreFX builds!
Please take a look at the doc change to find out more.
/cc @jorive @stephentoub @danmosemsft @AndreyAkinshin