Question

Two questions:

1. Can someone point me to unbiased data that compares .NET performance to VB 6 performance? I have searched but it is surprisingly difficult to find.
2. What is the best way to compare .NET performance to VB 6 performance as an app behaves at a customer’s site?

We have a WindowsForms, client-server app (written for 2.0, upgrading to 3.5 SP 1 soon) about which certain customers complain of ‘slow performance’ as compared to the previous VB 6 version. I know, ‘slow performance’ is very vague and general, but is it true to assume .NET code might be slower than VB 6 code because .NET runs in a VM? I wrote 100% of the code in C#, so it was not ported by some third person or wizard.

Not all customers make this complaint, so we suspect something environmental. Is our only option to measure performance at a customer site? Some of our customers use SQL Server 2005 on Windows Server 2003 on a Novell network. Would they see dramatically different data access performance than a similar machine on a Windows network?

Answer 1

The first thing I need to mention is that .Net code never runs in a VM. While it’s true .Net programs are compiled to IL that is somewhat analogous to Java’s ByteCode, the important difference is that the IL is also in turn compiled to fully native code before the app starts, rather than interpreted by a VM.

Moving on to the .Net/VB6 comparison. I can’t point to specific data, but from personal experience it depends on what you’re doing.

To illustrate this, let’s think about six different benchmark applications, each with a vb6 and .Net version. Each application picks out a specific operation, performs it 100,000 times, and then records the amount of time taken. Note that this is a thought experiment: I haven’t actually seen results from real apps. But I feel like I have a sense of the strengths and weaknesses of both platforms.

• Application A does some serious cpu-heavy number crunching. In this case, I have to believe the results here would be almost identical. Both VB6 and .Net compile down to native code, and so a cpu mult instruction is the same regardless. That said, if you’re not using Option Strict you could quickly get yourself in trouble on either platform. Since you used C# (which essentially is always Option Strict), that could give your .Net code an advantage.

• Application B goes out and retrieves a value from a database. Again, results are probably very close, though I’d have to give .Net a very slight edge for two reasons: it uses a native sql client, which is supposedly a little faster, and it does things like automatic connection pooling and makes it easier to factor out things like connecting once and re-using that connection rather than connecting repeatedly. But if you compare apples to apples as far as code goes, the two will likely be very close.

• Application C shows and hides a form repeatedly. Here I would have to give vb6 the nod. It’s based on an older, less glitzy widget set which will frankly be cheaper to render. Also, the WinForms components aren’t known for being all that speedy. However, the difference probably isn’t as large as you might think, and you’re probably not completely redrawing forms that often, either. This is likely the slowness that your users complain about.

• Application D is based on string operations, and here I have to give the nod to .Net. Both VB6 and .Net are known for slow string operations. However, .Net provides a number of tools that are simply not available in vb6 to help developers overcome the slowness. That said, if you’re not aware of those tools, poor string operation choices could easily bog down your app doing useless work. This is also likely contributing to user complaints.

• Application E is going to repeatedly load an Xml document into memory. Again, I have to think that .Net would have an advantage. First of all the xml tools available to vb were primitive at best. I find it unlikely they haven’t been improved significantly. Additionally, .Net’s garbage collector is pretty smart, giving it an advantage or allocated and freeing memory during the loads. However, I think the biggest thing here is going to be disk speed, which means the difference wouldn’t be that large.

• Application F will repeatedly start a .Net or vb6 program, wait for it to become ready (for some definition of ‘ready’), and then close it. Here vb6 likely has the advantage, for two reasons. Firstly, the .Net has to compile the IL to bytecode on the first run. Thankfully, that’s only the first run. Seconly, the .Net app also has to load any referenced assemblies. By comparison, the VB6 runtime is much smaller. However, this is normally only true the first time you start the app on a particular machine, and there are ways to pre-compile the .Net code. This may also contribute to perceived slowness.

One important point is that for all of these applications, the .Net app is likely going to have a much larger memory footprint. For some reasons developers tend to think about this as a bad performance characteristic when in fact the opposite is true. If your app is using more memory, it’s probably spending less time going to disk, and the disk is slower by far. It’s probably also caching more, which saves cpu time. And for .Net specifically, it’s saving up allocation/deallocation operations for periods when it’s more important or when the pc is otherwise idle.

I don’t have time, but it’s be interesting to see someone use the StopWatch implementation (at least for the .Net side) mentioned in today’s Coding Horror to get real benchmarks for each of these.