BenchmarkDotNet v0.10.0
- Now BenchmarkDotNet is a part of .NET Foundation
- Job and Column API refactoring (see new documentation)
- Measurement engine improvements
- Horology enhancement (see
TimeInterval
andFrequency
) - Introduced
RankColumn
which is based onWelchTTest
(see 157aabc3) - JsonExporters refactoring (see the Exporters/Json section in the documentation)
- Renamed JsonExporters classed and attributes
- JsonExporters with custom settings
- JsonExporters now includes information about the target type namespace (see #246).
- Add
JetBrains.Annotations
(see #253) - RFC 4180 support in CSV exporters (see #241)
- Many bugfixes
Milestone details
In the v0.10.0 scope, 19 issues were resolved and 7 pull requests were merged. This release includes 85 commits by 7 contributors.
Resolved issues (19)
- #30 Better information when we haven't got a valid measurement
- #121 Strange # of "Launches" chosen with Count.Auto
- #154 PathTooLong exception on custom config
- #185 Report if difference between 2 benchmarks is statistically significance
- #241 .csv results output does not play well with Excel or Google Sheets
- #244 DefaultConfig StatisticColumn values
- #246 No namespace information?
- #265 Add ability to specify that benchmark requires STAThread
- #266 Don't assume that TargetType has reference to BenchmarkDotNet (assignee: @adamsitnik)
- #268 Print runtime of child processes in summary
- #271 Params attribute does not handle nullable types
- #272 [Setup] error when doing inheritance
- #276 System.EntryPointNotFoundException
- #280 Cannot run on OSX / Mono (System.Xml.XmlException: Root element is missing)
- #281 Results are exported twice for single run
- #288 IdleWarmup running off forever
- #291 [Bug] Incorrect results for targetCount:Auto
- #292 Support for Beta versions
- #296 [BUG] NRE in OutliersAnalyser
Merged pull requests (7)
- #253 Mark [Benchmark] as implying implicit use (by @roji)
- #267 Make shipped assemblies have CLSCompliant(true) applied (by @lahma)
- #277 Changed diagnosers flow, reduced heap allocations in Engine to 0 (by @adamsitnik)
- #278 Support Atlassian flavored wiki markup (by @lahma)
- #286 Mutable Job implementation (by @ig-sinicyn)
- #287 Fix docs: job API changed (by @ig-sinicyn)
- #293 Presenters: IFormattable support (by @ig-sinicyn)
Commits (85)
- a8b4e7 JsonExporters refactoring (by @AndreyAkinshin)
- e6a4ae JsonExporters: add information about namespaces, resolves #246 (by @AndreyAkinshin)
- 29ee0a Add Namespace column (by @AndreyAkinshin)
- 26c333 Better error message (by @arthrp)
- 12b313 Merge pull request #251 from arthrp/master (by @AndreyAkinshin)
- 03d00c Mark [Benchmark] as implying implicit use (by @roji)
- 394a93 Merge pull request #253 from roji/resharper-annotations (by @AndreyAkinshin)
- 4912ea Remove unnecessary specific shell reference (by @factormystic)
- 023115 Merge pull request #254 from factormystic/patch-1 (by @AndreyAkinshin)
- 972fee Big refactoring (by @AndreyAkinshin)
- cf839a Improved ranks (by @AndreyAkinshin)
- ef3ecf Extended TimeInterval and Frequency API (by @AndreyAkinshin)
- dfcc98 Minor fixes (by @AndreyAkinshin)
- fb3757 Improved CSV export, fixes #241 (by @AndreyAkinshin)
- f490d1 further root folder cleanup #228 (by @adamsitnik)
- f7a6a0 Added [MeansImplicitUse] and explanation to summary for SetupAttribute and Cl... (by @adamsitnik)
- 56b1f2 Configuration fix in JitOptimizationsTests (by @AndreyAkinshin)
- faac58 Introduce AnaylyzeLaunchVariance (by @AndreyAkinshin)
- 9953a9 Smart statistics in StatisticsColumnProvider (by @AndreyAkinshin)
- 2a6578 Minor fixes in docs (by @AndreyAkinshin)
- 03fb04 Introduce UnrollFactor (by @AndreyAkinshin)
- 137636 NewLine fix in OutputLogger (by @AndreyAkinshin)
- b35d52 Unique column support (by @AndreyAkinshin)
- 24e944 Don't assume that TargetType has reference to BenchmarkDotNet, fixes #266 (by @adamsitnik)
- 69330a Make shipped assemblies have CLSCompliant(true) applied (by @lahma)
- fb8402 Merge pull request #267 from lahma/features/cls-compliancy (by @adamsitnik)
- 0944b5 Make exported HTML valid, add alternating color to result table (by @lahma)
- 748a2c Merge pull request #269 from lahma/features/html-export-enhancements (by @adamsitnik)
- afff51 ExecutionValidator should not throw on overridden [Setup] methods, fixes #272 (by @adamsitnik)
- ab9db4 Support nullable types as [Params], fixes #271 (by @adamsitnik)
- ce4994 support for [STAThread], fixes #265 (by @adamsitnik)
- 13fa5c move ConsoleHandler to separate file for better readability (by @adamsitnik)
- 0e8e82 Print runtime of child processes in summary, fixes #268 (by @adamsitnik)
- 491a28 Print more info about runtime of child processes in summary (and in a nicer w... (by @adamsitnik)
- 0580a5 updated docs for #265 (by @adamsitnik)
- f9baa6 catch native exceptions when determining clock type, #276 (by @adamsitnik)
- 93a23d Check OS version in WindowsClock, fixed #276 (by @AndreyAkinshin)
- 8d65fe changed diagnosers flow, possibility to hook up before jitting, after setup a... (by @adamsitnik)
- f346ff added predefined attributes for diagnosers (by @adamsitnik)
- e7cc6b Support Atlassian flavored wiki markup (by @lahma)
- 376bd8 reduce memory allocated by Engine during run to increase MemoryDiagnoser accu... (by @adamsitnik)
- b5ab55 display results when runing when no diagnoser is attached (by @adamsitnik)
- 7ae2b5 Improved idle method for primitive types (by @AndreyAkinshin)
- e1213a Fix NRE in BuildJobRuntimes (by @AndreyAkinshin)
- 572fa4 Make code CLS-Compliant, fix CS3015 warning (by @AndreyAkinshin)
- 4070a1 preload all settings in ctors, introduced IEngineFactory (by @adamsitnik)
- 2f0df7 make Engine use InvocationCount (by @adamsitnik)
- 8a7a12 calculate Statistics without allocations! makes the code look bad, but saves ... (by @adamsitnik)
- d8fa70 remove last allocations from Engine.Run (by @adamsitnik)
- f2a106 perform Jitting after first Setup call, better naming, test fix (by @adamsitnik)
- be3ce3 introduce general catch with hopes to help with #280 (by @adamsitnik)
- 3c5e70 post code review changes (by @adamsitnik)
- 16e258 possibility to define custom Engine (by @adamsitnik)
- afa586 Merge pull request #277 from PerfDotNet/diagnosersFlow (by @adamsitnik)
- 534189 export files only once, not twice, fixes #281 (by @adamsitnik)
- a733b5 Mutable Job implementation (by @ig-sinicyn)
- 259647 Merge pull request #286 from ig-sinicyn/feature-mutable-characteristics (by @AndreyAkinshin)
- c184b8 Fix docs: job API changed (by @ig-sinicyn)
- 47b152 Merge pull request #287 from ig-sinicyn/fix-docs-jobs-updated (by @AndreyAkinshin)
- 868ffc Jobs: WithXxx() extension methods added back, docs updated (by @ig-sinicyn)
- eede54 Jobs: .With() methods now create new instances of the Job. (by @ig-sinicyn)
- 40f160 Test fixed (by @ig-sinicyn)
- fd92f7 Jobs: helper for .With() methods (by @ig-sinicyn)
- 307b72 Merge pull request #289 from ig-sinicyn/features-jobs-with (by @AndreyAkinshin)
- d576f5 Presenters: IFormattable support (by @ig-sinicyn)
- 8647e7 Fix #291 (by @AndreyAkinshin)
- 49c000 Merge pull request #293 from ig-sinicyn/fix-presenter-culture (by @AndreyAkinshin)
- 684334 Merge pull request #278 from lahma/features/atlassian-wiki-markup (by @AndreyAkinshin)
- 7f3ca5 Make BenchmarkRunnerCore.Run public (by @AndreyAkinshin)
- 10a91e Allow using newer versions of Microsoft.NETCore.App, fixes #292 (by @adamsitnik)
- 134d74 Misc improvements in Exporters (by @AndreyAkinshin)
- 33c63f Respect RemoveOutliers in the RunResults (by @AndreyAkinshin)
- 5e022b Fix #291, part 2 (by @AndreyAkinshin)
- 4b6ea9 Improved name for the Id CharacteristicColumn (by @AndreyAkinshin)
- af6f8a Fix some compilation warnings (by @AndreyAkinshin)
- 2f8584 Analysers refactoring + OutliersAnalyser (by @AndreyAkinshin)
- b11935 Make BenchmarkRunnerCore public (by @AndreyAkinshin)
- 781740 Specify generated id for the default job (by @AndreyAkinshin)
- 846255 JobTests.Test01Create fix (by @AndreyAkinshin)
- 9f9544 Check if there is no MainTarget measurements in OutliersAnalyser, fixes #296 (by @AndreyAkinshin)
- 503b04 Proper exception in GetStatistics for empty input, see #296 (by @AndreyAkinshin)
- c667aa Fix path to logo (by @AndreyAkinshin)
- 0b91c3 Fix null check in JobMode.ApplyCore (by @AndreyAkinshin)
- 2edb56 Copyrights and links update (by @AndreyAkinshin)
- 2edb9a Set library version: 0.10.0 (by @AndreyAkinshin)
Contributors (7)
- Adam Sitnik (@adamsitnik)
- Andrey Akinshin (@AndreyAkinshin)
- Arthur (@arthrp)
- factormystic (@factormystic)
- ig-sinicyn (@ig-sinicyn)
- Marko Lahma (@lahma)
- Shay Rojansky (@roji)
Thank you very much!
Additional details
Date: November 10, 2016
Milestone: v0.10.0
NuGet Packages:
- https://www.nuget.org/packages/BenchmarkDotNet/0.10.0
- https://www.nuget.org/packages/BenchmarkDotNet.Core/0.10.0
- https://www.nuget.org/packages/BenchmarkDotNet.Toolchains.Roslyn/0.10.0
- https://www.nuget.org/packages/BenchmarkDotNet.Diagnostics.Windows/0.10.0
Online Documentation: https://dotnet.github.io/BenchmarkDotNet/