0.14.0

Full changelog: https://benchmarkdotnet.org/changelog/v0.14.0.html

Highlights

• Introduce BenchmarkDotNet.Diagnostics.dotMemory #2549: memory allocation profile of your benchmarks using dotMemory, see @BenchmarkDotNet.Samples.IntroDotMemoryDiagnoser
• Introduce BenchmarkDotNet.Exporters.Plotting #2560: plotting via ScottPlot (initial version)
• Multiple bugfixes
• The default build toolchains have been updated to pass IntermediateOutputPath, OutputPath, and OutDir properties to the dotnet build command. This change forces all build outputs to be placed in a new directory generated by BenchmarkDotNet, and fixes many issues that have been reported with builds. You can also access these paths in your own .csproj and .props from those properties if you need to copy custom files to the output.

Bug fixes

• Fixed multiple build-related bugs including passing MsBuildArguments and .Net 8's UseArtifactsOutput.

Breaking Changes

• DotNetCliBuilder removed retryFailedBuildWithNoDeps constructor option.
• DotNetCliCommand removed RetryFailedBuildWithNoDeps property and BuildNoRestoreNoDependencies() and PublishNoBuildAndNoRestore() methods (replaced with PublishNoRestore()).

0.13.12

Full changelog: https://benchmarkdotnet.org/changelog/v0.13.12.html

Highlights

The biggest highlight of this release if our new VSTest Adapter, which allows to run benchmarks as unit tests in your favorite IDE!
The detailed guide can be found here.

This release also includes to a minor bug fix that caused incorrect job id generation: fixed job id generation (#2491).

Also, the target framework in the BenchmarkDotNet templates was bumped to .NET 8.0.

0.13.11

Full changelog: https://benchmarkdotnet.org/changelog/v0.13.11.html

In the v0.13.11 scope, 4 issues were resolved and 8 pull requests were merged. This release includes 28 commits by 7 contributors.

Resolved issues (4)

• #2060 NativeAOT benchmark started from .Net Framework host doesn't have all intrinsics enabled (assignee: @adamsitnik)
• #2233 Q: Include hardware counters in XML output (assignee: @nazulg)
• #2388 Include AVX512 in listed HardwareIntrinsics
• #2463 Bug. Native AOT .NET 7.0 doesn't work. System.NotSupportedException: X86Serialize (assignee: @adamsitnik)

Merged pull requests (8)

• #2412 Add HardwareIntrinsics AVX-512 info (by @nietras)
• #2458 Adds Metrics Columns to Benchmark Report Output (by @nazulg)
• #2459 Enable MemoryDiagnoser on Legacy Mono (by @MichalPetryka)
• #2462 update SDK to .NET 8 (by @adamsitnik)
• #2464 Use "native" for .NET 8, don't use "serialize" for .NET 7 (by @adamsitnik)
• #2465 fix NativeAOT toolchain and tests (by @adamsitnik)
• #2468 Add OperationsPerSecondAttribute (by @DarkWanderer)
• #2475 Fix some tests (by @timcassell)

Commits (28)

• bb55e6 Set next BenchmarkDotNet version: 0.13.11 (by @AndreyAkinshin)
• db4d8b Adds Metrics Columns to Benchmark Report Output (#2458) (by @nazulg)
• e93b2b Use "native" for .NET 8, don't use "serialize" for .NET 7 (#2464) (by @adamsitnik)
• 127157 [build] Fix spellcheck-docs workflow (by @AndreyAkinshin)
• 8a02ec [build] Use our .NET SDK on Windows (by @AndreyAkinshin)
• 1b39e8 Suppress NU1903 in IntegrationTests.ManualRunning.MultipleFrameworks (by @AndreyAkinshin)
• e90311 update SDK to .NET 8 (#2462) (by @adamsitnik)
• fc7afe Enable MemoryDiagnoser on Legacy Mono (#2459) (by @MichalPetryka)
• 630622 fix NativeAOT toolchain and tests (#2465) (by @adamsitnik)
• 536a28 Add HardwareIntrinsics AVX-512 info (#2412) (by @nietras)
• 3fa045 Add OperationsPerSecondAttribute (#2468) (by @DarkWanderer)
• 0583cb Bump Microsoft.NETCore.Platforms: 5.0.0->6.0.0 (by @AndreyAkinshin)
• 2e62b9 Remove netcoreapp2.0;net461 from TFMs for IntegrationTests.ManualRunning.Mult... (by @AndreyAkinshin)
• 92fa3f Bump xunit: 2.5.0->2.6.2 (by @AndreyAkinshin)
• 01e220 Bump xunit.runner.visualstudio: 2.5.0->2.5.4 (by @AndreyAkinshin)
• 29a94c Bump Verify.Xunit: 20.3.2->20.8.2 (by @AndreyAkinshin)
• 538e0e Bump Microsoft.NET.Test.SDK: 17.6.2->17.8.0 (by @AndreyAkinshin)
• 136e4b Remove explicit Microsoft.NETFramework.ReferenceAssemblies reference in Bench... (by @AndreyAkinshin)
• 423b84 [build] Bump Docfx.App: 2.71.1->2.74.0 (by @AndreyAkinshin)
• 718953 [build] Bump Octokit: 7.0.0->9.0.0 (by @AndreyAkinshin)
• 0cce91 [build] Bump Cake.Frosting: 3.2.0->4.0.0 (by @AndreyAkinshin)
• 4d5dc9 Fix Newtonsoft.Json v13.0.1 in BenchmarkDotNet.IntegrationTests (by @AndreyAkinshin)
• c7ec60 Enable UserCanSpecifyCustomNuGetPackageDependency test on Linux (by @AndreyAkinshin)
• a572db Bump C# LangVersion: 11.0->12.0 (by @AndreyAkinshin)
• b4ac9d Nullability cleanup (2023-11-26) (by @AndreyAkinshin)
• 5557ae [build] Bump Docfx.App: 2.74.0->2.74.1 (by @AndreyAkinshin)
• b987b9 Fixed some tests. (#2475) (by @timcassell)
• 05eb00 Prepare v0.13.11 changelog (by @AndreyAkinshin)

Contributors (7)

• Adam Sitnik (@adamsitnik)
• Andrey Akinshin (@AndreyAkinshin)
• Michał Petryka (@MichalPetryka)
• Nazul Grimaldo (@nazulg)
• nietras (@nietras)
• Oleg V. Kozlyuk (@DarkWanderer)
• Tim Cassell (@timcassell)

Thank you very much!

0.13.10

Full changelog: https://benchmarkdotnet.org/changelog/v0.13.10.html

Highlights

Initial support of .NET 9 and minor bug fixes.

Details

In the v0.13.10 scope,
2 issues were resolved and 3 pull requests were merged.
This release includes 10 commits by 4 contributors.

Resolved issues (2)

• #2436 BenchmarkDotNet Access Denied Error on WSL2 when Writing to '/mnt/c/DumpStack.log.tmp' (assignee: @AndreyAkinshin)
• #2455 .NET 9 support (assignee: @adamsitnik)

Merged pull requests (3)

• #2447 Add support for wasm/net9.0 (by @radical)
• #2453 feat: set RuntimeHostConfigurationOption on generated project (by @workgroupengineering)
• #2456 implement full .NET 9 support (by @adamsitnik)

Commits (10)

• c27152 Set next BenchmarkDotNet version: 0.13.10 (by @AndreyAkinshin)
• 2e96d2 Don't show AssemblyInformationalVersion metadata in BDN BrandVersion (by @AndreyAkinshin)
• d17c6a Support Windows 11 23H2 (10.0.22631) in OsBrandStringHelper (by @AndreyAkinshin)
• af9c5c Exception handling in DotNetCliGenerator.GetRootDirectory, fix #2436 (by @AndreyAkinshin)
• e11136 [build] Bump Docfx.App: 2.71.0->2.71.1 (by @AndreyAkinshin)
• 7b342f Add support for wasm/net9.0 (#2447) (by @radical)
• e17068 Adjust 'Failed to set up high priority' message (by @AndreyAkinshin)
• 0a734a feat: set RuntimeHostConfigurationOption on generated project (#2453) (by @workgroupengineering)
• ae4914 implement full .NET 9 support (#2456) (by @adamsitnik)
• 40c414 Prepare v0.13.10 changelog (by @AndreyAkinshin)

Contributors (4)

• Adam Sitnik (@adamsitnik)
• Andrey Akinshin (@AndreyAkinshin)
• Ankit Jain (@radical)
• workgroupengineering (@workgroupengineering)

Thank you very much!

0.13.9

Full changelog: https://benchmarkdotnet.org/changelog/v0.13.9.html

In the v0.13.9 scope, 3 issues were resolved and 7 pull requests were merged. This release includes 26 commits by 5 contributors.

Resolved issues (3)

• #2054 Custom logging/visualization during the benchmark run (assignee: @caaavik-msft)
• #2404 Using DisassemblyDiagnoser in GitHub Actions (assignee: @timcassell)
• #2432 Something went wrong with outliers when using --launchCount (assignee: @AndreyAkinshin)

Merged pull requests (7)

• #1882 use coalesce instead of join (by @askazakov)
• #2413 Fix linux crash from disassembler (by @timcassell)
• #2420 Add event processor functionality (by @caaavik-msft)
• #2421 More nullability warnings fixes (by @alinasmirnova)
• #2433 Fix build errors with latest sdk (by @timcassell)
• #2434 Fix Event Processors not being copied in ManualConfig.Add (by @caaavik-msft)
• #2435 Treat warnings not as errors in manual test project (by @timcassell)

Commits (26)

• ece5cc Set next BenchmarkDotNet version: 0.13.9 (by @AndreyAkinshin)
• ad9376 Add event processor functionality (#2420) (by @caaavik-msft)
• 8227bb Address PR feedback (#2434) (by @caaavik-msft)
• 46b3c0 Fix linux crash from disassembler (#2413) (by @timcassell)
• 967a97 Fix build errors with latest sdk. (#2433) (by @timcassell)
• dd7a9b Treat warnings not as errors in manual test project (#2435) (by @timcassell)
• 583874 Print full stacktrace for GenerateException, see #2436 (by @AndreyAkinshin)
• 6e3a15 Support WSL detection in RuntimeInformation (by @AndreyAkinshin)
• 8986e0 Update myget url in README (by @AndreyAkinshin)
• 516bd6 Enabled nullability for BenchmarkDotNet.Diagnostics.dotTrace.csproj (by @alinasmirnova)
• 5428eb Fixed nullability warnings in methods signatures (by @alinasmirnova)
• 7fbbc9 Removed CanBeNull attribute (by @alinasmirnova)
• 9d7350 Fixed warnings on null assignments (by @alinasmirnova)
• b43d28 Fixed warnings in EngineEventLogParser (by @alinasmirnova)
• 148165 Removed an unnecessary check (by @alinasmirnova)
• 465aaf Fixed empty catch warning (by @alinasmirnova)
• 9a7bb7 [build] Bump: Microsoft.DocAsCode.App 2.67.5 -> Docfx.App 2.71.0 (by @AndreyAkinshin)
• 9dd7b6 Fix license badge link in README (by @AndreyAkinshin)
• 134b8e [build] Automatic NextVersion evaluation (by @AndreyAkinshin)
• 8090d9 Suppress NETSDK1138 (TFM out of support warning) (by @AndreyAkinshin)
• af610e Bump .NET SDK: 7.0.305->7.0.401 (by @AndreyAkinshin)
• 8838ed [build] Fix docfx build warnings (by @AndreyAkinshin)
• 2d379b Remove outlier consistency check, fix #2432 (by @AndreyAkinshin)
• 00628a use coalesce instead of join (by @askazakov)
• 411a6e Prepare v0.13.9 changelog (by @AndreyAkinshin)
• 228a46 Rollback docfx.json (by @AndreyAkinshin)

Contributors (5)

• Alina Smirnova (@alinasmirnova)
• Andrey Akinshin (@AndreyAkinshin)
• askazakov (@askazakov)
• Cameron Aavik (@caaavik-msft)
• Tim Cassell (@timcassell)

Thank you very much!

0.13.8

Full changelog: https://benchmarkdotnet.org/changelog/v0.13.8.html

Highlights

This release contains important bug fixes.

What's Changed

• Issue2394 multiple markdown exporters not possible even with different names by @bstordrup in #2395
• Make MarkdownExporter ctor and Dialect protected by @nietras in #2407
• Refactor out base TextLogger from StreamLogger by @nietras in #2406
• • update the templates install command to reflect dotnet cli updates by @baywet in #2415
• Update stub decoding for .NET 8 for disassemblers by @janvorli in #2416
• Enable nullability for BenchmarkDotNet.Annotations by @alinasmirnova in #2418
• Nullability In BenchmarkDotNet project by @alinasmirnova in #2419
• feat: add text justification style by @Vahdanian in #2410
• Default to RoslynToolchain by @timcassell in #2409

New Contributors

• @bstordrup made their first contribution in #2395
• @baywet made their first contribution in #2415
• @Vahdanian made their first contribution in #2410

Full Changelog: v0.13.7...v0.13.8

BenchmarkDotNet v0.13.7

This release contains a bunch of important bug fixes.

Full changelog: https://benchmarkdotnet.org/changelog/v0.13.7.html

What's Changed

• Improve build for mono aot by @radical in #2367
• IComparable fallback for Tuple/ValueTuple by @mrahhal in #2368
• Don't copy PackageReference in csproj by @timcassell in #2365
• Fix regression in parsing arguments with spaces Closes #2373 by @kant2002 in #2375
• AggressiveOptimization in InProcess toolchains by @timcassell in #2335
• Add expected results tests by @timcassell in #2361
• [chore]: fix error message by @BurakTaner in #2379
• Cancel old jobs on push by @timcassell in #2380
• Support --cli argument for CsProjClassicNetToolchain by @timcassell in #2381
• Rebuild .Net Framework projects by @timcassell in #2370
• Fix missing import on Debug build by @caaavik-msft in #2385
• perfcollect: don't restore symbols for local builds by @adamsitnik in #2384
• Fix PlatformNotSupportedException thrown on Android in ConsoleTitler by @Adam-- in #2390

New Contributors

• @BurakTaner made their first contribution in #2379
• @caaavik-msft made their first contribution in #2385
• @Adam-- made their first contribution in #2390

Full Changelog: v0.13.6...v0.13.7

BenchmarkDotNet v0.13.6

Highlights

• New BenchmarkDotNet.Diagnostics.dotTrace NuGet package. Once this package is installed, you can annotate your benchmarks with the [DotTraceDiagnoser] and get a dotTrace performance snapshot at the end of the benchmark run. #2328
• Updated documentation website. We migrated to docfx 2.67 and got the refreshed modern template based on bootstrap 5 with dark/light theme switcher.
• Updated BenchmarkDotNet.Templates. Multiple issues were resolved, now you can create new benchmark projects from terminal or your favorite IDE. #1658 #1881 #2149 #2338
• Response file support. Now it's possible to pass additional arguments to BenchmarkDotNet using @filename syntax. #2320 #2348
• Custom runtime support. #2285
• Introduce CategoryDiscoverer, see IntroCategoryDiscoverer. #2306 #2307
• Multiple bug fixes.

Full changelog: https://benchmarkdotnet.org/changelog/v0.13.6.html