BenchmarkDotNet v0.15.0

Milestone details

In the v0.15.0 scope, 25 issues were resolved and 43 pull requests were merged. This release includes 62 commits by 25 contributors.

Resolved issues (25)

#1067 Conditions in Directory.Build.props fail (assignee: @timcassell)
#1120 Large Spike from WorkloadWarmup to WorkloadActual (assignee: @timcassell)
#1738 Async GlobalSetup does not work with InProcess toolchain (assignee: @timcassell)
#2236 Benchmarks sporadically failing on Executable Emitted.dll not found (assignee: @timcassell)
#2383 DisassemblyDiagnoser does not work in InProcessEmitToolchain (assignee: @timcassell)
#2411 Jobs can mess up builds with --keepFiles (assignee: @timcassell)
#2425 WithCustomBuildConfiguration breaks build (assignee: @timcassell)
#2483 Pressing Ctrl-C does not reset power plan back
#2516 Add validation warning for sealed classes containing benchmarks (assignee: @ketanpkolte)
#2520 If a test run fails, the report shows it passed. (assignee: @ketanpkolte)
#2533 MonoAOTLLVM run failed to build properly but still ran (assignee: @timcassell)
#2577 Processor model is not detected on Azure arm64 (assignee: @AndreyAkinshin)
#2591 Not possible to compute the (Ratio, RatioSD)
#2599 Generic benchmarks in one report
#2611 ScottPlotExporter is incorrectly calculating mean on measurements from all iteration stages (assignee: @FlatlinerDOA)
#2629 Feature Request: Add ability to automatically hide metric columns if value is not set
#2638 Missing disassembly on .NET 9 (assignee: @timcassell)
#2652 Build problems with BenchmarkDotNet 0.14.0 and Grpc.Tools 2.67.0 (assignee: @timcassell)
#2654 [Documentation] console-args.md is out of date (assignee: @5AIPAVAN)
#2664 Starting CLI Commands no longer works via the dotnet-runtime-host ".exe" (assignee: @timcassell)
#2677 Release changes for #2651 (assignee: @AndreyAkinshin)
#2678 Getting assembly conflict warnings on net471 after updating to 0.14.0 from 0.13.12 (assignee: @adamsitnik)
#2701 InProcessNoEmitToolchain + NativeAOT Support?
#2712 BenchmarkDotNet should warn if there are no benchmarks (assignee: @AvishaiDotan)
#2721 Feature Request: Add options to disable build in parallel.

Merged pull requests (43)

#2109 Fix async GlobalSetup/Cleanup with InProcessEmit toolchain (by @timcassell)
#2423 Fix builds when --keepFiles is specified (by @timcassell)
#2488 Update clrmd to 3.1 (by @timcassell)
#2505 Remove support for old core runtimes (by @timcassell)
#2535 Always fail if dotnet command failed (by @timcassell)
#2579 Support required properties (by @aelij)
#2612 Fix #2611 - Add Iteration mode and stage filtering to ScottPlotExporter (by @FlatlinerDOA)
#2613 Fix known high severity vulnerabilities (by @adamsitnik)
#2614 Feature: Box Plot support for ScottPlotExporter (by @FlatlinerDOA)
#2616 Clean up unsupported native AOT flags (by @MichalStrehovsky)
#2618 Update good-practices.md (by @SteveDunn)
#2622 Fix deadlock in GetDotNetSdkVersion (by @Korexio)
#2626 Bump Perfolizer + Initial Phd adoption (by @AndreyAkinshin)
#2627 Refactor dotTrace and dotMemory diagnosers (by @AndreyAkinshin)
#2642 Add .NET 10 support (by @am11)
#2643 Add DOTNET_ environment vars (by @am11)
#2645 Change .NET SDK Validator to account for backwards compatibility (by @caaavik-msft)
#2646 Get full parsable version part in .NET SDK Validator (by @caaavik-msft)
#2647 add RiscV64 support (by @adamsitnik)
#2650 improve BaselineCustomAnalyzer warning message, fixes #2591 (by @znxftw)
#2653 Fixed GitHub workflow report for failed tests (by @ketanpkolte)
#2657 Update console-args.md (by @5AIPAVAN)
#2660 Fixed: Add validation warning for sealed classes containing benchmarks (by @ketanpkolte)
#2661 Prevent premature end of the Benchmark process at Ctrl-C, fixes #2483 (by @leonvandermeer)
#2665 Native AOT projects should also copy SettingsWeWantToCopy (by @keegan-caruso)
#2667 Remove obsolete API usage in articles (by @keegan-caruso)
#2670 Implement WakeLock so that the system does not enter sleep (by @leonvandermeer)
#2672 Add documentation for VS Profiler (by @karpinsn)
#2673 Feature Request: Add ability to automatically hide metric columns if value is not set (by @AvishaiDotan)
#2674 add HostSignal.AfterProcessStart to allow the users to obtain ID of a process that was started suspended (by @adamsitnik)
#2676 Use ArtifactsPath (by @timcassell)
#2680 Cleanup templates (by @timcassell)
#2681 Fix Windows path too long (by @timcassell)
#2686 Some minor fixes before new version release (by @adamsitnik)
#2688 Constant stack size (by @timcassell)
#2692 Revert remove old runtimes (by @timcassell)
#2702 Basic support of InProcessNoEmitRunner for NativeAOT. (by @eliphatfs)
#2703 Missing dotnet 9.0 support. (by @VAllens)
#2709 Exporters.Plotting: Move to ScottPlot 5.0.54 (by @bclehmann)
#2716 Allow to set ParamsAttribute's values from derived class (by @filzrev)
#2718 Log a warning when an empty benchmark is run (by @AvishaiDotan)
#2725 feat: Add option to disable parallel build (by @filzrev)
#2731 Added Measurements to DiagnoserResults (by @persn)

Commits (62)

a739e2 Set next BenchmarkDotNet version: 0.14.1 (by @AndreyAkinshin)
c8105a Fix deadlock in GetDotNetSdkVersion (#2622) (by @Korexio)
29a0eb Update good-practices.md (#2618) (by @SteveDunn)
3927ce Fix #2611 - Add Iteration mode and stage filtering to ScottPlotExporter (#2612) (by @FlatlinerDOA)
cd9e4e Feature - Added Box plots (by @FlatlinerDOA)
d2f73e Bump .NET SDK: 8.0.101->8.0.401 (by @AndreyAkinshin)
64b3d8 Refactor CpuInfo detection, fix #2577 (by @AndreyAkinshin)
5e9b35 Fix lscpu cpu frequency parsing (by @AndreyAkinshin)
a58872 Make lscpu call language-invariant, fix #2577 (by @AndreyAkinshin)
ae4bb9 Bump Perfolizer + Initial Phd adoption (by @AndreyAkinshin)
296c99 Remove ConfigCompatibilityValidator, fix #2599 (by @AndreyAkinshin)
92f33f Refactor dotTrace and dotMemory diagnosers (by @AndreyAkinshin)
475ab4 Move Smoke benchmarks to BenchmarkDotNet.Samples (by @AndreyAkinshin)
ca5dfd Clean up unsupported native AOT flags (#2616) (by @MichalStrehovsky)
bc3abf Update clrmd to 3.1 (#2488) (by @timcassell)
3a2d11 Removed support for netcoreapp3.0 and older runtimes. (#2505) (by @timcassell)
adf8e6 Add DOTNET_ environment vars (#2643) (by @am11)
5fe0c7 Change .NET SDK Validator to account for backwards compatibility (#2645) (by @caaavik-msft)
52485e Get full parsable version part (#2646) (by @caaavik-msft)
9040e4 add RiscV64 support, fixes #2644 (#2647) (by @adamsitnik)
9ff49c Fixed GitHub workflow report for failed tests (#2653) (by @ketanpkolte)
1f7cab Update console-args.md (#2657) (by @5AIPAVAN)
6248e8 improve baseline warning message (#2650) (by @znxftw)
af8bde Fix known high severity vulnerabilities (#2613) (by @adamsitnik)
346bba Fixed: Add validation warning for sealed classes containing benchmarks (#2660) (by @ketanpkolte)
b9d69a Prevent premature end of the Benchmark process at Ctrl-C, fixes #2483 (#2661) (by @leonvandermeer)
25308b Native AOT projects should also copy SettingsWeWantToCopy (#2665) (by @keegan-caruso)
c7ed71 Remove obsolete API usage in articles (#2667) (by @keegan-caruso)
5f0c47 Fix async GlobalSetup/GlobalCleanup not being awaited with InProcessEmit too... (by @timcassell)
6367ad Add documentation for VS Profiler (#2672) (by @karpinsn)
fe5b2f Fix builds when --keepFiles is specified (#2423) (by @timcassell)
cd50f7 add HostSignal.AfterProcessStart to allow the users to obtain ID of a process... (by @adamsitnik)
cac4f6 Feature Request: Add ability to automatically hide metric columns if value is... (by @AvishaiDotan)
3337a0 Add .NET 10 support (#2642) (by @am11)
1aab1c Add support for user-supplied project file detection (#2684) (by @Genbox)
390463 Some minor fixes before new version release (#2686) (by @adamsitnik)
804482 Revert "Add support for user-supplied project file detection (#2684)" (#2687) (by @adamsitnik)
de8fb9 Implement WakeLock so that the system does not enter sleep (#2670) (by @leonvandermeer)
e7e758 Revert remove old runtimes (#2692) (by @timcassell)
021fc6 Missing dotnet 9.0 support. (#2703) (by @VAllens)
6ce979 Basic support of InProcessNoEmitRunner for NativeAOT. (#2702) (by @eliphatfs)
1c62b2 Fix URLs in documentation (#2705) (by @WojciechNagorski)
ce6eb7 Exporters.Plotting: Move to ScottPlot 5.0.54 (#2709) (by @bclehmann)
c4bc75 Support required properties (#2579) (by @aelij)
30c730 Add test for required properties in InProcess (#2713) (by @aelij)
21da33 chore: allow to set ParamsAttribute values from derived class (#2716) (by @filzrev)
f4bfcd Fix publish-nightly failures (#2695) (by @timcassell)
8af544 Bump Perfolizer: 0.4.0->0.4.1 (needed for #2688) (by @AndreyAkinshin)
49514c [build] Bump BenchmarkDotNet.Build dependencies (by @AndreyAkinshin)
250f9b Bump dawidd6/action-download-artifact from 2 to 6 in /.github/workflows (by @dependabot[bot])
42c32f Removed unnecessary output path properties from csproj templates. (#2680) (by @timcassell)
602c5f [build] Update InTestsCoreTask description (by @AndreyAkinshin)
43c7b5 Log a warning when an empty benchmark is run (#2718) (by @AvishaiDotan)
924ca6 Always fail if dotnet command failed. (#2535) (by @timcassell)
ee248c feat: Add option to disable parallel build (#2725) (by @filzrev)
0a5c82 [docs] Prepare v0.15.0 changelog (by @AndreyAkinshin)
3afc4e Use ArtifactsPath (#2676) (by @timcassell)
8b14cb Constant stack size (#2688) (by @timcassell)
5045ea Added Measurements to DiagnoserResults (#2731) (by @persn)
c7cbbf Bump Perfolizer: 0.4.1->0.5.2 (and update API usage) (by @AndreyAkinshin)
56610b Fix Windows path too long (#2681) (by @timcassell)
4c839d Update README and template.json files (by @AndreyAkinshin)

Contributors (25)

Adam Sitnik (@adamsitnik)
Adeel Mujahid (@am11)
Allen Cai (@VAllens)
Andrew Chisholm (@FlatlinerDOA)
Andrey Akinshin (@AndreyAkinshin)
Avishai Dotan (@AvishaiDotan)
Benjamin Lehmann (@bclehmann)
Cameron Aavik (@caaavik-msft)
dependabot[bot] (@dependabot[bot])
Eli Arbel (@aelij)
filzrev (@filzrev)
Ian Qvist (@Genbox)
Keegan (@keegan-caruso)
Ketan Pramod Kolte (@ketanpkolte)
Korexio (@Korexio)
leonvandermeer (@leonvandermeer)
Michal Strehovský (@MichalStrehovsky)
Nik Karpinsky (@karpinsn)
Per Olav Flaten (@persn)
Ruoxi (@eliphatfs)
Saipavan Lingamallu (@5AIPAVAN)
Steve Dunn (@SteveDunn)
Tim Cassell (@timcassell)
Vishnu Bhagyanath (@znxftw)
Wojciech Nagórski (@WojciechNagorski)

Thank you very much!

Additional details

Date: May 22, 2025

Milestone: v0.15.0 (List of commits)

NuGet Packages:

Table of Contents

BenchmarkDotNet v0.15.0

Milestone details

Resolved issues (25)

Merged pull requests (43)

Commits (62)

Contributors (25)

Additional details