BenchmarkDotNet v0.13.6

Highlights

• New BenchmarkDotNet.Diagnostics.dotTrace NuGet package. Once this package is installed, you can annotate your benchmarks with the [DotTraceDiagnoser] and get a dotTrace performance snapshot at the end of the benchmark run. #2328
• Updated documentation website. We migrated to docfx 2.67 and got the refreshed modern template based on bootstrap 5 with dark/light theme switcher.
• Updated BenchmarkDotNet.Templates. Multiple issues were resolved, now you can create new benchmark projects from terminal or your favorite IDE. #1658 #1881 #2149 #2338
• Response file support. Now it's possible to pass additional arguments to BenchmarkDotNet using @filename syntax. #2320 #2348
• Custom runtime support. #2285
• Introduce CategoryDiscoverer, see IntroCategoryDiscoverer. #2306 #2307
• Multiple bug fixes.

Milestone details

In the v0.13.6 scope, 19 issues were resolved and 33 pull requests were merged. This release includes 127 commits by 14 contributors.

Resolved issues (19)

• #1658 Pass arguments to runner in dotnet new templates
• #1694 Length cannot be less than zero
• #1783 MemoryDiagnoser table output invalid symbol
• #1856 Not Set Hardware Counter throws Argument Null Exception
• #1877 Any combination of Params/ParamsSource/ParamsAllValues is broken
• #1881 broken template for Rider
• #2036 Publish changelog for each BenchmarkDotNet versions using GitHub releases (assignee: @AndreyAkinshin)
• #2149 BenchmarkDotNet.Templates not showing in VS2022(17.3.6) new project dialog
• #2249 BenchmarkSwitcher.Run nullability check warnings in .NET 7 (assignee: @AndreyAkinshin)
• #2259 CSproj code generator produces invalid syntax after incorrectly parsing .targets file
• #2264 Benchmarkdotnet crashes when flushing the logger
• #2266 Incompatible Microsoft.CodeAnalysis.CSharp versioning with Razor
• #2271 'Unknown processor' yet again (assignee: @br3aker)
• #2279 Endless compile errors (assignee: @AndreyAkinshin)
• #2282 TimeUnit Setting Not Being Used in Excel Report
• #2306 Allow for benchmark categories composition using inheritance (assignee: @AndreyAkinshin)
• #2312 CodeAnalysisTreatWarningsAsErrors=true in Directory.Build.props breaks auto-generated benchmark assembly build (assignee: @viktorz)
• #2333 Cannot benchmark netcoreapp3.0
• #2358 System.MissingMethodException: Method not found: 'System.String Perfolizer.Horology.TimeInterval.ToString(System.Globalization.CultureInfo, System.String, Perfolizer.Common.UnitPresentation)'

Merged pull requests (33)

• #1879 adjust defaultBenchmarkDotNetVersion on build (by @askazakov)
• #2086 Tell the user about the asterisk (by @YegorStepanov)
• #2106 If a metric value could not be calculated, display ? instead of -. (by @timcassell)
• #2124 Fix crash when using HardwareCounter.NotSet (by @YegorStepanov)
• #2141 Add support for nested generic types (by @YegorStepanov)
• #2272 Try to shorten the log file name to avoid TooLongPath on Windows (by @AndreyAkinshin)
• #2274 Add Android specific GetNetCoreVersion Runtime Information capabilities. (by @LoopedBard3)
• #2276 Fix nullability issues in runner method signatures, fix #2249 (by @AndreyAkinshin)
• #2277 Bump xunit 2.4.1->2.4.2 (by @AndreyAkinshin)
• #2278 lock all access to CompositeLogger (by @adamsitnik)
• #2280 Add ParamsValidator (by @AndreyAkinshin)
• #2283 Improve ParamsValidator (by @AndreyAkinshin)
• #2285 Allow custom runtimes to use Executor (by @tgjones)
• #2286 Use config summary style in CsvExporter by default, fix #2282 (by @AndreyAkinshin)
• #2288 update Microsoft.CodeAnalysis.CSharp, fixes #2266 (by @dennishermann)
• #2300 Fix typos in good-practices.md (by @mrahhal)
• #2302 Fix net8.0 tfm detection (by @adamsitnik)
• #2304 IComparable for complex params (by @mrahhal)
• #2307 Introduce CategoryDiscoverer, fix #2306 (by @AndreyAkinshin)
• #2316 Updated CsProj.txt, set CodeAnalysisTreatWarningsAsErrors=false (by @viktorz)
• #2320 Add support for response files (by @kant2002)
• #2328 Add BenchmarkDotNet.Diagnostics.dotTrace (by @AndreyAkinshin)
• #2329 Bump Microsoft.NET.Test.Sdk: 16.11.0->17.6.2 (by @AndreyAkinshin)
• #2330 Use Verify library for approval tests (by @alinasmirnova)
• #2338 Improve templates (by @YegorStepanov)
• #2339 Add the args parameter to RunAll and RunAllJoined (by @YegorStepanov)
• #2343 Fix JitOptimizationsValidator when ran in NativeAOT (by @timcassell)
• #2347 Copy PackageReferences to generated csproj (by @timcassell)
• #2348 Add docs for RSP file support (by @kant2002)
• #2352 Fix netcoreapp3.0 and older builds (by @timcassell)
• #2359 Fix netcoreapp3.0 builds (by @timcassell)
• #2360 Fix MosCpuInfoProvider (by @timcassell)
• #2362 Cancel old CI jobs (by @timcassell)

Commits (127)

• 401030 Postrelease v0.13.5 update (by @AndreyAkinshin)
• e2de0e Try to shorten the log file name to avoid TooLongPath on Windows (#2272) (by @AndreyAkinshin)
• fb7f1b Add Android specific NetCoreVersion. (#2274) (by @LoopedBard3)
• 6c128c Fix nullability issues in runner method signatures, fix #2249 (by @AndreyAkinshin)
• d9ecba Bump xunit 2.4.1->2.4.2 (by @AndreyAkinshin)
• cb571d Move ChangeLogBuilder.cs to the build project (by @AndreyAkinshin)
• c0b592 Remove scripts in docs (by @AndreyAkinshin)
• b26471 Bump docfx: 2.59.4 -> 2.62.1 (by @AndreyAkinshin)
• 0772ce log all access to CompositeLogger (#2278) (by @adamsitnik)
• 5148fe ReflectionExtensions cleanup (by @AndreyAkinshin)
• e07363 Add ParamsValidator for readonly/const fields (see #2279) (by @AndreyAkinshin)
• 4b6e83 Add validation for multiple Params* attributes, fix #1877 (by @AndreyAkinshin)
• b846d5 Add validation for Params* attributes on init-only properties, fix #1877 (by @AndreyAkinshin)
• 975a0e Fix crash when using HardwareCounter.NotSet (#2124) (by @YegorStepanov)
• 6162e9 Improve HardwareCounters validation, see #1856 (by @AndreyAkinshin)
• 565870 Allow custom runtimes to use Executor (#2285) (by @tgjones)
• 2d04a2 Use config summary style in CsvExporter by default, fix #2282 (#2286) (by @AndreyAkinshin)
• 809fed Add user help about asterisk (#2086) (by @YegorStepanov)
• ec429a update Microsoft.CodeAnalysis.CSharp, fixes #2266 (#2288) (by @dennishermann)
• 9b3186 Add twitter badge in README (by @AndreyAkinshin)
• 56c66d Create FUNDING.yml (by @AndreyAkinshin)
• bfb192 Fix net8.0 tfm detection (#2302) (by @adamsitnik)
• 492e58 Improve ParamsValidator (#2283) (by @AndreyAkinshin)
• f32a2e IComparable for complex params (#2304) (by @mrahhal)
• 960289 Enabled category inheritance, fix #2306 (by @AndreyAkinshin)
• 521b7f Introduce CategoryDiscoverer, fix #2306 (by @AndreyAkinshin)
• 40305b Add support for nested generic types (#2141) (by @YegorStepanov)
• 572dd1 Updated CsProj.txt, set CodeAnalysisTreatWarningsAsErrors=false (#2316) (by @viktorz)
• 43815d Improve timeouts for InProcessEmitExecutor under profiling (by @AndreyAkinshin)
• 7c4241 Fix nullability issues in AssemblyExtensions (by @AndreyAkinshin)
• fc7490 Enable verbose xunit logs for CI tests (by @AndreyAkinshin)
• e1f1d4 Ignore "Platform linker not found" in local runs (by @AndreyAkinshin)
• 40dca9 Fix compilation (by @AndreyAkinshin)
• c3a0f8 Workaround for xunit+ApprovalTests integration issues (by @AndreyAkinshin)
• adf72c Fix MemoryDiagnoserSupportsNativeAOT (by @AndreyAkinshin)
• b1baa7 Better assert message for AllSetupAndCleanupMethodRunsTest (by @AndreyAkinshin)
• 775ad4 Increase FinishEventTimeout in AsyncProcessOutputReader (by @AndreyAkinshin)
• 567562 Fix local DisassemblyDiagnoserTests run on macOS (by @AndreyAkinshin)
• a260bd Improve 'command took' log line format (by @AndreyAkinshin)
• ff6e8d Add BenchmarkDotNet.Diagnostics.dotTrace (by @AndreyAkinshin)
• 1c5597 Fix links in docs (by @AndreyAkinshin)
• 4b265c Fix generation of the docs/api (by @AndreyAkinshin)
• bb46d8 Bump .NET SDK: 7.0.200->7.0.302 (by @AndreyAkinshin)
• 6553e0 Bump docfx: 2.62.1->2.67.3 (by @AndreyAkinshin)
• e2c321 Simplify api generation via cake+docfx (by @AndreyAkinshin)
• 88beb2 Fix building doc specifying .net 4.6.2 as requirement (by @JobaDiniz)
• 43ae33 Bump .NET SDK: 7.0.302->7.0.304 (security fix) (by @AndreyAkinshin)
• 862617 Make formatting DotSettings consistent with Roslyn analyzer (by @AndreyAkinshin)
• bfbcec Fix ConfigCompatibilityValidatorTests on Linux (by @AndreyAkinshin)
• bed071 Bump Microsoft.NET.Test.Sdk: 16.11.0->17.6.2 (by @AndreyAkinshin)
• 73f8fd Use Verify library for approval tests (#2330) (by @alinasmirnova)
• b725ac Fix PackTask.ShouldRun (by @AndreyAkinshin)
• 2e1774 Catch the NotSupportedException thrown in NativeAOT for `GetReferencedAssem... (by @timcassell)
• 58ee5c Add the args parameter to RunAll and RunAllJoined (#2339) (by @YegorStepanov)
• 977c05 Add support for response files (#2320) (by @kant2002)
• 17df12 Add docs for RSP file support (#2348) (by @kant2002)
• dc3cef Improve templates (#2338) (by @YegorStepanov)
• d6e080 Move RedirectGenerator to the build project (by @AndreyAkinshin)
• b797cf Update versioning in build scripts (by @AndreyAkinshin)
• ebe925 Revert "Update versioning in build scripts" (by @AndreyAkinshin)
• b91501 Update versioning in build scripts (by @AndreyAkinshin)
• 68bedd Update changelog builder (by @AndreyAkinshin)
• d46c32 Update contributors names and count in changelogs (by @AndreyAkinshin)
• 319f0e Remove hardcoded markdown from code, generate dynamically (by @AndreyAkinshin)
• 6d005a Update UID in changelog generation (by @AndreyAkinshin)
• fb6f43 Fix full.md generation (by @AndreyAkinshin)
• 124476 Bump Cake.FileHelpers: 6.0.0->6.1.3 (by @AndreyAkinshin)
• 0286c4 Split MsBuildSettings to separate tasks in the Build project (by @AndreyAkinshin)
• a519b6 Copy PackageReferences to generated csproj (#2347) (by @timcassell)
• 74c1e1 Added platform property to MsBuildSettingsBuild (by @AndreyAkinshin)
• 4e9edb Update Pack workflow (by @AndreyAkinshin)
• 9b8227 Move generated changelog details to a separate branch (by @AndreyAkinshin)
• 562a80 Add workflows/docs-changelog-generate.yaml (by @AndreyAkinshin)
• f519a5 Exclude 'docs-changelog-details' from builds (by @AndreyAkinshin)
• 152ad7 Update changelog download strategy in workflows (by @AndreyAkinshin)
• fbc0d7 Fix changelog downloading workflow (by @AndreyAkinshin)
• 748448 Add UpdateStats task (by @AndreyAkinshin)
• 609e48 Generate index.md from README.md (by @AndreyAkinshin)
• c4debe adjust defaultBenchmarkDotNetVersion on build (#1879) (by @askazakov)
• 88cb48 If a metric value could not be calculated, display ? instead of -. (#2106) (by @timcassell)
• 3af22c Prepare v0.13.6 changelog (by @AndreyAkinshin)
• 2d5156 Remove Ben.Demystifier from Build project (by @AndreyAkinshin)
• ee1ae7 Bump Octokit: 6.2.1->7.0.0 (by @AndreyAkinshin)
• dc54ac Bump docfx: 2.67.3->2.67.5 (by @AndreyAkinshin)
• 7986ce Remove excessive logs in cleanup (by @AndreyAkinshin)
• c352b3 Improve RScript discover strategy in RPlotExporter (by @AndreyAkinshin)
• 7dc098 Handle exceptions in GetDebuggableAttribute (by @AndreyAkinshin)
• 5065bc Bump C#: 9.0->11.0 (by @AndreyAkinshin)
• 48a7b6 Support Linux brand versions (by @AndreyAkinshin)
• 03ebe4 Simplify the environment info presentation (by @AndreyAkinshin)
• 0978d0 Support macOS 14 Sonoma in OsBrandStringHelper (by @AndreyAkinshin)
• a6edfe Fix CPU frequency detection on Linux (by @AndreyAkinshin)
• 144f5c Fix CPU detection on Windows when wmic is not available via PATH, fix #2271 (by @AndreyAkinshin)
• 6291a7 Huge build project refactoring (by @AndreyAkinshin)
• 47bd6d Bump JetBrains.Profiler.SelfApi: 2.4.2->2.5.0 (by @AndreyAkinshin)
• f5777b Disable net462 integration tests on non-windows os (by @AndreyAkinshin)
• c78b7d Fix race in Broker, workaround for #2317 (#2318) (by @AndreyAkinshin)
• 389908 Suppress dependency rebuilding in integration tests on windows+net7.0 (by @AndreyAkinshin)
• 469b43 Update test-reporter (by @AndreyAkinshin)
• 2b8bc8 Fix netcoreapp3.0 and older builds (#2352) (by @timcassell)
• c0cbd2 Add version history to BuildContext (by @AndreyAkinshin)
• 41f86e Rework documentation generation (by @AndreyAkinshin)
• 4243a6 Improve build script option parsing (by @AndreyAkinshin)
• 210020 Rework GitHub workflows (by @AndreyAkinshin)
• c5fd22 Fix DocsUpdate (by @AndreyAkinshin)
• b6cbad Fix report-test-results.yaml (by @AndreyAkinshin)
• 519537 report-test-results.yaml: fix artifact download (by @AndreyAkinshin)
• 81cf9f report-test-results.yaml: fix permissions (by @AndreyAkinshin)
• e42dae GitHub workflow rename (by @AndreyAkinshin)
• 554b00 Update report-test-results.yaml (by @AndreyAkinshin)
• f3a799 Update Github Actions workflow files (by @AndreyAkinshin)
• cca009 Update report-test-results.yaml (by @AndreyAkinshin)
• 28a196 Disable AppVeyor integration (by @AndreyAkinshin)
• bd89a0 Disable Azure Pipelines integration (by @AndreyAkinshin)
• 1e9c99 Update version handling in build properties (by @AndreyAkinshin)
• d4c70e Update version prefix to 0.13.6 (by @AndreyAkinshin)
• bf020e Automatically update changelog footer (by @AndreyAkinshin)
• 3b361e Add Release build task (by @AndreyAkinshin)
• f43495 Fix documentation generation (by @AndreyAkinshin)
• 30887e Update team.md (by @AndreyAkinshin)
• 28e003 Update license.md (by @AndreyAkinshin)
• d883fd Suppress running publish-nightly on forked repositories (by Andrey Akinshin)
• ca9d8b Fix MosCpuInfoProvider (#2360) (by @timcassell)
• 3e7334 Cancel old CI jobs when a new push is done. (#2362) (by @timcassell)
• 9171cf Lock perfolizer reference, fix #2358 (by @AndreyAkinshin)
• 278549 Add MyGet badge in README (by @AndreyAkinshin)
• e85fe2 Fix netcoreapp3.0 builds (#2359) (by @timcassell)

Contributors (14)

• Adam Sitnik (@adamsitnik)
• Alina Smirnova (@alinasmirnova)
• Andrey Akinshin
• Andrey Akinshin (@AndreyAkinshin)
• Andrii Kurdiumov (@kant2002)
• askazakov (@askazakov)
• dennishermann (@dennishermann)
• Joberto Diniz (@JobaDiniz)
• Mohammad Rahhal (@mrahhal)
• Parker Bibus (@LoopedBard3)
• Tim Cassell (@timcassell)
• Tim Jones (@tgjones)
• ViktorZ (@viktorz)
• Yegor Stepanov (@YegorStepanov)

Thank you very much!

Additional details

Date: July 11, 2023

Milestone: v0.13.6 (List of commits)

NuGet Packages:

• https://www.nuget.org/packages/BenchmarkDotNet/0.13.6
• https://www.nuget.org/packages/BenchmarkDotNet.Annotations/0.13.6
• https://www.nuget.org/packages/BenchmarkDotNet.Diagnostics.dotTrace/0.13.6
• https://www.nuget.org/packages/BenchmarkDotNet.Diagnostics.Windows/0.13.6
• https://www.nuget.org/packages/BenchmarkDotNet.Templates/0.13.6