How to benchmark C# code using BenchmarkDotNet


BenchmarkDotNet is a lightweight, open source, powerful .NET library that can transform your methods into benchmarks, track those methods, and then provide insights into the performance data captured. It is easy to write BenchmarkDotNet benchmarks and the results of the benchmarking process are user friendly as well.

You can take advantage of BenchmarkDotNet to benchmark both .NET Framework and .NET Core applications. In this article we’ll explore how we can work with BenchmarkDotNet in .NET Core. You can .

To work with the code examples provided in this article, you should have Visual Studio 2019 installed in your system. If you don’t already have a copy, you can . 

Create a console application project in Visual Studio

First off, let’s create a .NET Core console application project in Visual Studio. Assuming Visual Studio 2019 is installed in your system, follow the steps outlined below to create a new .NET Core console application project in Visual Studio.

  1. Launch the Visual Studio IDE.
  2. Click on “Create new project.”
  3. In the “Create new project” window, select “Console App (.NET Core)” from the list of templates displayed.
  4. Click Next.
  5. In the “Configure your new project” window shown next, specify the name and location for the new project.
  6. Click Create.

This will create a new .NET Core console application project in Visual Studio 2019.

Note that when you create the console application project, the resulting Program class (generated automatically in the Program.cs file) will look like this:


Figure 1. Running benchmark code in debug mode will result in an error. 

When benchmarking you should always ensure that you run your project in release mode. The reason is that during compilation the code is optimized differently for both debug and release modes. The C# compiler does a few optimizations in release mode that are not available in debug mode.

Hence you should run your project in the release mode only. To run benchmarking, specify the following command at the Visual Studio command prompt.

dotnet run -p BenchmarkDotNetDemo.csproj -c Release

For best results, you should make sure that all applications are closed and all unnecessary processes stopped prior to running benchmarks.

Note that if you don’t specify the configuration parameter then the runtime will attempt to do benchmarking on non-optimized, debug-mode code. And you’ll be presented with the same error shown in Figure 1.

Analyze the benchmarking results

Once the execution of the benchmarking process is complete, a summary of the results will be displayed at the console window. The summary section contains information related to the environment in which the benchmarks were executed, such as the BenchmarkDotNet version, operating system, computer hardware, .NET version, compiler information, and information related to the performance of the application.

A few files will also be created in the BenchmarkDotNet.Artifacts folder under the application’s root folder. Here is a summary of the results. 


Figure 2. Summary of BenchmarkDotNet benchmark results. 

As evident from the summary shown in Figure 2, for each benchmarked method, you will see a row of data that specifies the performance metrics such as mean execution time, Gen 0, Gen 1, Gen 2 collections, etc.

On examining the results shown in Figure 3, you can see that the ConcatStringUsingGenericList is much faster than the ConcatStringUsingStringBuilder method. You can also see that there are many more allocations after running the ConcatStringUsingStringBuilder method.


Figure 3. Another view of BenchmarkDotNet benchmark results. 

Now add the RankColumn attribute on top of the MemoryBenchmarkerDemo class. This will add an extra column to the output indicating which method was faster. Run the benchmarking process again using the following command.

dotnet run -p BenchmarkDotNetDemo.csproj -c Release

When you run this command, the benchmarking process kicks off and displays the output after the benchmarking process has been executed successfully. Figure 4 below shows the output with RankColumn added. 


Figure 4. BenchmarkDotNet benchmark results including the RankColumn attribute. 

BenchmarkDotNet is a nice tool that provides a simple way to make an informed decision about the performance metrics of your application. In BenchmarkDotNet, invocation of a method that has the Benchmark attribute set is known as an operation. An iteration is the name given to a collection of several operations. 

You can explore a demo ASP.NET Core application that illustrates several ways to benchmark the code. You can  on GitHub. 

How to do more in C#: 

Copyright © 2020 IDG Communications, Inc.