Question

I’m using Visual Studio 2010 SP1, Target framework is 2.0, Platform target: Any CPU, testing under Windows 7 x64 SP1.

I’m experiencing strange performance behavior.

Without an app.config, or with the following app.config, it makes my program run slowly (Stopwatch shows ~0.11 s)

<?xml version="1.0"?>
<configuration>
  <startup >
    <supportedRuntime version="v2.0.50727" />
  </startup>
</configuration>

The following app.config makes my program run x5 times faster (Stopwatch shows ~0.02 s)

<?xml version="1.0"?>
<configuration>
  <startup >
    <supportedRuntime version="v4.0.30319" sku=".NETFramework,Version=v4.0" />
  </startup>
</configuration>

This is the test program code:

using System;
using System.Collections.Generic;
using System.Text;
using System.Diagnostics;

class Program
{
    static void Main(string[] args)
    {
        Stopwatch sw = new Stopwatch();

        while (true)
        {
            sw.Reset();
            sw.Start();

            for (int i = 0; i < 1000000; i++ )
            {
                "blablabla".IndexOf("ngrhotbegmhroes", StringComparison.OrdinalIgnoreCase);
            }

            Console.WriteLine(sw.Elapsed);
        }
    }
}

I’m sitting for hours and can’t figure out what is happening here.
Have you any idea?

Answer 1

It sounds like you’ve just found a situation in which .NET 4 is a lot faster. By default, your app is running with the framework it was built to target. When you force it to use .NET 4, it’s faster. That may be a JIT compiler improvement which happens to hit your situation, or it may be a framework improvement – but it shouldn’t be too surprising that some things are faster in newer versions.

(For what it’s worth, I’d increase the number of iterations you’re timing over if I were you… on my box under .NET 4, each iteration is only 10ms, which isn’t really a great measurement. I prefer to benchmark for at least a few seconds.)

(And like Mitch, I can confirm that I see the same effect.)

EDIT: I’ve just investigated this a bit further, and seen an interesting effect… I’ll assume we’re calling haystack.IndexOf(needle, StringComparison.OrdinalIgnoreCase):

• On .NET 2, the results are roughly the same however big the “needle” is
• On .NET 4:
  • If needle is bigger than haystack (as per your example) .NET 4 is much faster than .NET 2
  • If needle is the same size as haystack, .NET 4 is a little bit slower than .NET 2
  • If needle is smaller than haystack, .NET 4 is a lot slower than .NET 2

(This is keeping a test where the first character of needle never appears in haystack, btw.)