Table of Contents

Setup And Cleanup

Sometimes we want to write some logic which should be executed before or after a benchmark, but we don't want to measure it. For this purpose, BenchmarkDotNet provides a set of attributes: [GlobalSetup], [GlobalCleanup], [IterationSetup], [IterationCleanup].

Sample: IntroSetupCleanupGlobal

A method which is marked by the [GlobalSetup] attribute will be executed only once per a benchmarked method after initialization of benchmark parameters and before all the benchmark method invocations. A method which is marked by the [GlobalCleanup] attribute will be executed only once per a benchmarked method after all the benchmark method invocations. If you are using some unmanaged resources (e.g., which were created in the GlobalSetup method), they can be disposed in the GlobalCleanup method.

Source code

using BenchmarkDotNet.Attributes;

namespace BenchmarkDotNet.Samples
{
    public class IntroSetupCleanupGlobal
    {
        [Params(10, 100, 1000)]
        public int N;

        private int[] data;

        [GlobalSetup]
        public void GlobalSetup()
        {
            data = new int[N]; // executed once per each N value
        }

        [Benchmark]
        public int Logic()
        {
            int res = 0;
            for (int i = 0; i < N; i++)
                res += data[i];
            return res;
        }

        [GlobalCleanup]
        public void GlobalCleanup()
        {
            // Disposing logic
        }
    }
}

Sample: IntroSetupCleanupIteration

A method which is marked by the [IterationSetup] attribute will be executed exactly once before each benchmark invocation, forcing UnrollFactor=1 and InvocationCount=1 (we have changed that in 0.11.0). It's not recommended to use this attribute in microbenchmarks because it can spoil the results. However, if you are writing a macrobenchmark (e.g. a benchmark which takes at least 100ms) and you want to prepare some data before each invocation, [IterationSetup] can be useful.

A method which is marked by the [IterationCleanup] attribute will be executed exactly once after each invocation. This attribute has the same set of constraint with [IterationSetup]: it's not recommended to use [IterationCleanup] in microbenchmarks.

Source code

using System;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Engines;

namespace BenchmarkDotNet.Samples
{
    [SimpleJob(RunStrategy.Monitoring, launchCount: 1,
        warmupCount: 2, iterationCount: 3)]
    public class IntroSetupCleanupIteration
    {
        private int setupCounter;
        private int cleanupCounter;

        [IterationSetup]
        public void IterationSetup()
            => Console.WriteLine($"// IterationSetup ({++setupCounter})");

        [IterationCleanup]
        public void IterationCleanup()
            => Console.WriteLine($"// IterationCleanup ({++cleanupCounter})");

        [GlobalSetup]
        public void GlobalSetup()
            => Console.WriteLine("// " + "GlobalSetup");

        [GlobalCleanup]
        public void GlobalCleanup()
            => Console.WriteLine("// " + "GlobalCleanup");

        [Benchmark]
        public void Benchmark()
            => Console.WriteLine("// " + "Benchmark");
    }
}

The order of method calls

// GlobalSetup

// IterationSetup (1)    // IterationSetup Jitting
// IterationCleanup (1)  // IterationCleanup Jitting

// IterationSetup (2)    // MainWarmup1
// Benchmark             // MainWarmup1
// IterationCleanup (2)  // MainWarmup1

// IterationSetup (3)    // MainWarmup2
// Benchmark             // MainWarmup2
// IterationCleanup (3)  // MainWarmup2

// IterationSetup (4)    // MainTarget1
// Benchmark             // MainTarget1
// IterationCleanup (4)  // MainTarget1

// IterationSetup (5)    // MainTarget2
// Benchmark             // MainTarget2
// IterationCleanup (5)  // MainTarget2

// IterationSetup (6)    // MainTarget3
// Benchmark             // MainTarget3
// IterationCleanup (6)  // MainTarget3

// GlobalCleanup

Sample: IntroSetupCleanupTarget

Sometimes it's useful to run setup or cleanups for specific benchmarks. All four setup and cleanup attributes have a Target property that allow the setup/cleanup method to be run for one or more specific benchmark methods.

Source code

using System;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Engines;

namespace BenchmarkDotNet.Samples
{
    [SimpleJob(RunStrategy.Monitoring, launchCount: 0,
        warmupCount: 0, iterationCount: 1)]
    public class IntroSetupCleanupTarget
    {
        [GlobalSetup(Target = nameof(BenchmarkA))]
        public void GlobalSetupA()
            => Console.WriteLine("// " + "GlobalSetup A");

        [Benchmark]
        public void BenchmarkA()
            => Console.WriteLine("// " + "Benchmark A");

        [GlobalSetup(Targets = new[] { nameof(BenchmarkB), nameof(BenchmarkC) })]
        public void GlobalSetupB()
            => Console.WriteLine("// " + "GlobalSetup B");

        [Benchmark]
        public void BenchmarkB()
            => Console.WriteLine("// " + "Benchmark B");

        [Benchmark]
        public void BenchmarkC()
            => Console.WriteLine("// " + "Benchmark C");

        [Benchmark]
        public void BenchmarkD()
            => Console.WriteLine("// " + "Benchmark D");
    }
}

The order of method calls

// GlobalSetup A

// Benchmark A

// GlobalSetup B

// Benchmark B

// GlobalSetup B

// Benchmark C

// Benchmark D