BenchmarkDotNet v0.10.4

New logo
Update to Roslyn 2.0, drop .NET 4.5 support (#303)
Initial support of HardwareCounters (Windows only)
Initial experimental support of in-process benchmarks
Optional configs for BenchmarkSwitcher (#391, #392)
Host API interface (#356)
Improved measurements for async benchmarks (#415)
Improved precision level (MinIterationTimes is 500ms instead of 200ms; introduced AccuracyMode.MaxAbsoluteError and AccuracyMode.MaxRelativeError instead of AccuracyMode.MaxStdErrRelative; logic which select amount of iterations uses confidence intervals instead of standard errors; the Error column (half of CI99.9%) is shown by default instead of StdErr)
Introduced ISummaryStyle, raw data in CSV reports (#118, #146, #396)
Handle cases when report files are existed and locked (#414, #416)
MarkdownExporter right-justifies numeric columns (#421)
Better colors for console output (#376)
Column legends
Add information about CPU microarchitecture for well-known processors to summary
Fix AssemblyInformationalVersionAttribute (#382)
Fix incorrect method filtering in BenchmarkSwitcher (#365)
Fix OS Version in Summary for Windows 10 (#351)
Fix OS Version on Mono
Fix --class and --method filtering (#249)
Fix --exporters option (#189)
Fix escaping logic in CsvExporter (#294, #409)
Fix MacOS detection
Minor bugfixes and API improvements

Milestone details

In the v0.10.4 scope, 23 issues were resolved and 14 pull requests were merged. This release includes 103 commits by 9 contributors.

Resolved issues (23)

#118 Raw data in CSV reports (assignee: @AmadeusW)
#146 Ability to specify units / easier comparison (assignee: @AmadeusW)
#159 Warn user if no Columns were defined (assignee: @adamsitnik)
#189 --exporters option appears not to be working (assignee: @adamsitnik)
#249 --class and --method should combine as "AND" filtering (assignee: @adamsitnik)
#294 [Suggestion] CSVHelper.Escape() method should check for actual separator value (assignee: @alinasmirnova)
#303 Update to Roslyn 2.0 when RTM is shipped to nuget.org (assignee: @adamsitnik)
#351 Fix OS Version in Summary for Windows 10 (assignee: @adamsitnik)
#352 Troubles with CoreJob on Linux (assignee: @adamsitnik)
#365 [Minor bug] Benchmark switcher: incorrect method filtering (assignee: @adamsitnik)
#376 Pick better background colors for output (assignee: @AndreyAkinshin)
#382 AssemblyInformationalVersion doesn't work (assignee: @AndreyAkinshin)
#388 Precise Machine Counter Diagnoser (assignee: @adamsitnik)
#391 BenchmarkSwitcher should take an optinal IConfig
#393 Troubles with ClrJob in .NET Core applications (assignee: @adamsitnik)
#395 Could not load file or assembly 'System.Reflection.Metadata' (assignee: @adamsitnik)
#401 Exceptions in Roslyn.Builder (assignee: @adamsitnik)
#406 BenchmarkDotNet with netcoreapp2.0 requires using RuntimeFrameworkVersion directly in the project file (assignee: @adamsitnik)
#410 Troubles with Classic applications on nightly BenchmarkDotNet (assignee: @adamsitnik)
#412 HardwareCounter.InstructionRetired failing with ArgumentNullException. Build 82 (assignee: @adamsitnik)
#415 Allocations for async methods measures BenchmarkDotNet (assignee: @adamsitnik)
#419 Suspicious warnings about MemoryMappedFiles (assignee: @adamsitnik)
#424 Make InliningDiagnoser filtering more flexible (assignee: @adamsitnik)

Merged pull requests (14)

#356 Feature: host API interface (by @ig-sinicyn)
#379 Feature: in-process benchmarks (by @ig-sinicyn)
#392 Add an optional config to BenchmarkSwitcher. (by @ILMTitan)
#396 Allow users to pick, show and hide measurement units in the reports and exports. (by @AmadeusW)
#400 fix spelling error: misspredict => mispredict (by @SteveDesmond-ca)
#405 Make RoslynToolchain types public (by @cdmihai)
#407 Propagate benchmark to hooks (by @cdmihai)
#408 Small fixes before approval tests (by @alinasmirnova)
#409 CSVHelper.Escape() method should check for actual separator value (by @alinasmirnova)
#416 Exports file to temporary location if target is locked (by @AmadeusW)
#421 MarkdownExporter right-justifies numeric columns (by @SteveDesmond-ca)
#423 Updated Microsoft.Net.Test.Sdk (by @alinasmirnova)
#430 Fix unix OS detection. (by @mfilippov)
#432 Fix macOS detection (by @mfilippov)

Commits (103)

bd790c Simplify BenchmarkDotNet.IntegrationTests.Classic.ReferencesTests (by @AndreyAkinshin)
bf9f4e Feature: host API interface (by @ig-sinicyn)
f85ffe Merge pull request #356 from ig-sinicyn/feature-host-api (by @AndreyAkinshin)
8ffff0 Feature: in-process benchmarks (by @ig-sinicyn)
51fc0c Feature: in-process benchmarks, FixAffinity() helper (by @ig-sinicyn)
c5179f Feature: in-process benchmarks, review fixes (by @ig-sinicyn)
6c5990 Feature: in-process benchmarks, review fixes, part 2 (by @ig-sinicyn)
c6b046 Feature: in-process benchmarks, STA & priority on full .Net FW (by @ig-sinicyn)
cc2c89 Feature: in-process benchmarks, env validation in toolchain, intro example ad... (by @ig-sinicyn)
101a80 Improved versioning system (by @AndreyAkinshin)
846532 Update links to appveyor (by @AndreyAkinshin)
5464e4 Improved versioning system, part 2 (by @AndreyAkinshin)
e39a50 Merge pull request #379 from ig-sinicyn/feature-inprocess (by @adamsitnik)
ae4cc8 AssemblyInformationalVersion fix in common.props, resolves #382 (by @AndreyAkinshin)
d036f9 Precise Machine Counter Diagnoser #388 (by @adamsitnik)
390442 Improved invocationCount behavior in SimpleJobAttribute (by @AndreyAkinshin)
7b61ed add possibility to set Hardware Counters per class in runtime-independent way... (by @adamsitnik)
6e208c Add an optional config to BenchmarkSwitcher. (by ILMTitan)
935d23 Merge pull request #392 from ILMTitan/master (by @adamsitnik)
6b9a88 update to Roslyn 2.0, drop .NET 4.5 support, fixes #303 (by @adamsitnik)
ea4c55 BenchmarkDotNet.IntegrationTests.csproj: fix formatting (by @AndreyAkinshin)
20a537 Update year in LICENSE.md (by @AndreyAkinshin)
878796 Update logo (by @AndreyAkinshin)
faba77 Update README.md (by @AndreyAkinshin)
6dce74 Update logo in README.md (by @AndreyAkinshin)
7a750f Another fix in README.md (by @AndreyAkinshin)
1caa0d fix spelling error: misspredict => mispredict (by @SteveDesmond-ca)
742912 use Array.Empty and Task.Completed (after upgrading to .NET 4.6) (by @adamsitnik)
0dafac there is no need to set up with latest VS, fixes #393 (by @adamsitnik)
f5d88e Catch exceptions in BenchmarkDotNet.Running.BenchmarkRunnerCore.Run (by @AndreyAkinshin)
fa176f Merge pull request #400 from stevedesmond-ca/missspelling (by @AndreyAkinshin)
90c3b1 Docs improvements (by @AndreyAkinshin)
84c524 Mark SimpleJobAttribute with AllowMultiple (by @AndreyAkinshin)
531804 Update README.md (by @AndreyAkinshin)
7f2221 Make RoslynToolchain types public (#405) (by @cdmihai)
be8c33 Improved confidence intervals (by @AndreyAkinshin)
767811 Propagate benchmark to hooks (#407) (by @cdmihai)
ec5e54 All exporters should not use static environment info (by @alinasmirnova)
2b8a9e Culculate csv separator when it is needed, not in constructor (by @alinasmirnova)
95aeb5 clean up (by @alinasmirnova)
2e74c2 Name field for all exporters (by @alinasmirnova)
badb0b Merge pull request #408 from alinasmirnova/small-fixes (by @AndreyAkinshin)
9da9ca search for .NET Core 2.0 settings in imported props files, fixes #406 (by @adamsitnik)
2b08c5 filter sealed, generic and abstract classes from BenchmarkSwitcher, fixes #365 (by @adamsitnik)
14ad55 print correct OS version in summary, fixes #351 (by @adamsitnik)
b86f48 BenchmarkDotNet does not support running .NET Core benchmarks when host proce... (by @adamsitnik)
11b51b refactoring in TypeParser (by @adamsitnik)
8047a7 use AND when filtering with args from command line, fixes #249 (by @adamsitnik)
3cf8d8 handle cmd line arguments without '=', fixes #189 (by @adamsitnik)
cdc71f test fix after recent changes, #249 (by @adamsitnik)
97c211 detect situation when users want to use Hardware Counters with InProcessToolc... (by @adamsitnik)
ba972b Warn user if no Columns were defined, fixes #159 (by @adamsitnik)
2520f2 migrate old csprojs to the new format to get the tests running in common way,... (by @adamsitnik)
dd1b4d Improvements in StatisticsTests (by @AndreyAkinshin)
07d149 emptyEnumerable.All(whatever) returns true (by @adamsitnik)
0d61a7 improve dynamic diagnoser loading (case when diagnosers NuGet pacakge is inst... (by @adamsitnik)
5a6937 Ignore BenchmarkProject.json in BenchmarkDotNet.sln.DotSettings (by @AndreyAkinshin)
5895c4 Improvements in TypeParserTests (by @AndreyAkinshin)
1124ae BenchmarkDotNet.Tests: cleanup (by @AndreyAkinshin)
d37ef4 CSVHelper.Escape() method should check for actual separator value (by @alinasmirnova)
43643c move hardwareCounters from Job to Config, fixes #412 (by @adamsitnik)
303fff pass config from runner => executor => diagnoser #412 (by @adamsitnik)
5b432e Addded approval tests for exporters (by @alinasmirnova)
ac6507 Added more info in mock summary (by @alinasmirnova)
a59b17 MockEnvironmentInfo for approval tests (by @alinasmirnova)
610f3b Awaiting Tasks should not interfere allocation results, fixes #415 (by @adamsitnik)
cf16f6 I forgot about aligning, #415 (by @adamsitnik)
2860d2 help the .NET framework to resolve assemblies when binding redirects are miss... (by @adamsitnik)
e7ad36 don't log false alarms, fixes #419 (by @adamsitnik)
803081 Allow users to pick, show and hide measurement units in the reports and expor... (by @AmadeusW)
fa3128 Exports file to temporary location if target is locked (#416) (by @AmadeusW)
c923ba MarkdownExporter right-justifies numeric columns (#421) (by @SteveDesmond-ca)
aa290d Add column legends (by @AndreyAkinshin)
8d8db5 Handle case when there are no columns with legends (by @AndreyAkinshin)
dc201c Add empty line before legends (by @AndreyAkinshin)
ff7fc9 Approved files (by @alinasmirnova)
3f6372 Updated Microsoft.Net.Test.Sdk (by @alinasmirnova)
7d23d1 Merge remote-tracking branch 'upstream/master' (by @alinasmirnova)
57cbbb Merge branch 'new-test-sdk-version' (by @alinasmirnova)
c0c563 Updated Microsoft.Net.Test.Sdk (by @alinasmirnova)
539834 Updated spproved files according last changes in md exporter (by @alinasmirnova)
f117f0 Added *.received and *.orig in .gitignore (by @alinasmirnova)
0d7e72 Make InliningDiagnoser filtering more flexible, fixes #424 (by @adamsitnik)
d25eb0 allow to set InProcessToolchain via attribute /cc @ig-sinicyn (by @adamsitnik)
8cf041 Mock strings in mock environment info (by @alinasmirnova)
0db804 Changed file naming in approval files (by @alinasmirnova)
29aeaf correct Idle implementation for Task-returning benchmarks, fixes #418 (by @adamsitnik)
c81aa5 the missing docs for Hardware Counters, fixes #388 (by @adamsitnik)
6076e2 Improved precision (by @AndreyAkinshin)
72b3a6 Show "NA" for statistic columns with double.NaN values (by @AndreyAkinshin)
ea2222 Processor brand string prettifying (by @AndreyAkinshin)
678d43 Added approval files info links to docs (by @alinasmirnova)
33911c Merge remote-tracking branch 'upstream/master' (by @alinasmirnova)
d4c928 Fixed approval tests according to recent changes in master (by @alinasmirnova)
eb9f4f Fix typos in docs (by @AndreyAkinshin)
271b27 Merge pull request #348 from alinasmirnova/master (by @AndreyAkinshin)
152f3e Parsing Gulftown processor brand strings (by @AndreyAkinshin)
74d793 Minor improvements in docs (by @AndreyAkinshin)
764bd3 Minor improvements in docs, part 2 (by @AndreyAkinshin)
0a251b Fix unix OS detection. (by @mfilippov)
5c3b39 Fix typo in PlatformID (by @mfilippov)
7f3d06 Always use PlatformAbstractions in RuntimeInformation.GetOsVersion() (by @AndreyAkinshin)
46d34f Set library version: 0.10.4 (by @AndreyAkinshin)

Contributors (9)

Adam Sitnik (@adamsitnik)
Alina Smirnova (@alinasmirnova)
Amadeusz Wieczorek (@AmadeusW)
Andrey Akinshin (@AndreyAkinshin)
ig-sinicyn (@ig-sinicyn)
ILMTitan
Mihai Codoban (@cdmihai)
Mikhail Filippov (@mfilippov)
Steve Desmond (@SteveDesmond-ca)

Thank you very much!

Additional details

Date: April 21, 2017

Milestone: v0.10.4

NuGet Packages: