BenchmarkDotNet v0.15.2

Highlights

• The most significant update in this release is the enhanced accuracy of the memory diagnoser (#2562). This improvement resolves the issue of incorrectly reported memory allocations (#1542, #2582).
• We have introduced a new feature that allows users to sort benchmark jobs in numerical order (#2768, #2770).
• Benchmark validation has been improved (#2771).
• An issue with non-persistent auto-generated JobId has been fixed (#2777).

Milestone details

In the v0.15.2 scope, 5 issues were resolved and 10 pull requests were merged. This release includes 15 commits by 3 contributors.

Resolved issues (5)

• #1542 Unexpected allocations reported in CPU-bound code (assignee: @timcassell)
• #1925 Flaky tests because of AssertAllocations with InProcessEmitToolchain on Windows (assignee: @timcassell)
• #2582 IterationSetup allocates 400B (assignee: @timcassell)
• #2768 [Feature Request] Add custom option to sort benchmark jobs by numeric orders (assignee: @filzrev)
• #2779 AllocationQuantumIsNotAnIssueForNetCore21Plus tests randomly failed on macos (arm64)

Merged pull requests (10)

• #2562 Improve memory diagnoser accuracy (by @timcassell)
• #2766 chore: Fix benchmark TestAdapter related issues (by @filzrev)
• #2769 chore: Suppress xunit non-serializable data warnings (by @filzrev)
• #2770 feat: Add JobOrderPolicy option to sort jobs by numeric order (by @filzrev)
• #2771 feat: Add validator for benchmarks that contains null runtime (by @filzrev)
• #2772 chore: Add setting to skip test reports when run-tests workflow is cancelled (by @filzrev)
• #2775 chore: Modify macos runner image for CI (by @filzrev)
• #2777 fix: Issue that auto-generate JobId is changed between benchmark runs (by @filzrev)
• #2780 chore: Skip null runtime validation for InProcessToolchain (by @filzrev)
• #2782 chore: Add warmups steps for flaky tests (AllocationQuantumIsNotAnIssueForNetCore21Plus) (by @filzrev)

Commits (15)

• 19e907 Set next BenchmarkDotNet version: 0.15.2 (by @AndreyAkinshin)
• 200260 [build] Enhance docs-fetch command (by @AndreyAkinshin)
• 3df0f1 Remove docs/_changelog folder from the main branch (by @AndreyAkinshin)
• c9548f chore: fix benchmark testadapter related issues (#2766) (by @filzrev)
• c84de9 Improve memory diagnoser accuracy (#2562) (by @timcassell)
• fe3b31 chore: add setting to skip test reports when original workflow is cancelled (... (by @filzrev)
• 6730bf feat: add JobOrderPolicy option to sort jobs numeric order (#2770) (by @filzrev)
• a08f01 chore: Modify macos runner image for CI (#2775) (by @filzrev)
• de8ba0 chore: suppress xunit non-serializable data warnings (#2769) (by @filzrev)
• ed5316 feat: Add validator for benchmarks that contains null runtime (#2771) (by @filzrev)
• 5db728 fix: auto-generate jobid between benchmark runs (by @filzrev)
• 27d864 chore: fix flaky memory allocation test (#2782) (by @filzrev)
• 5f1dcd chore: skip null runtime validation for inprocess toolchain (#2780) (by @filzrev)
• 973f6b [build] Allow workflow_dispatch for publish-nightly workflow (by @AndreyAkinshin)
• 2306ba [build] Enable --force-clone for docs-fetch in generate-gh-pages.yaml (by @AndreyAkinshin)

Contributors (3)

• Andrey Akinshin (@AndreyAkinshin)
• filzrev (@filzrev)
• Tim Cassell (@timcassell)

Thank you very much!

Additional details

Date: June 16, 2025

Milestone: v0.15.2 (List of commits)

NuGet Packages:

• https://www.nuget.org/packages/BenchmarkDotNet/0.15.2
• https://www.nuget.org/packages/BenchmarkDotNet.Annotations/0.15.2
• https://www.nuget.org/packages/BenchmarkDotNet.Diagnostics.dotMemory/0.15.2
• https://www.nuget.org/packages/BenchmarkDotNet.Diagnostics.dotTrace/0.15.2
• https://www.nuget.org/packages/BenchmarkDotNet.Diagnostics.Windows/0.15.2
• https://www.nuget.org/packages/BenchmarkDotNet.Exporters.Plotting/0.15.2
• https://www.nuget.org/packages/BenchmarkDotNet.Templates/0.15.2
• https://www.nuget.org/packages/BenchmarkDotNet.TestAdapter/0.15.2