Question

Background (You can skip this section)

I have a large amount of data (about 3 mb) that needs to be kept up to date on several hundred machines. Some of the machines run C# and some run Java. The data could change at any time and needs to be propogated to the clients within minutes. The data is delivered in Json format from 4 load balanced servers. These 4 servers are running ASP.NET 4.0 with Mvc 3 and C# 4.0.

The code that runs on the 4 servers has a hashing algorithm which hashes the Json response and then converts the hash to a string. This hash is given to the client. Then, every few minutes, the clients ping the server with the hash and if the hash is out of date the new Json object is returned. If the hash is still current then a 304 with an emptry body is returned.

Occasionally the hashes generated by the 4 boxes are inconsistent across the boxes, which means that the clients are constantly downloading the data (each request could hit a different server).

Code Snipet

Here is the code that is used to generate the hash.

internal static HashAlgorithm Hasher { get; set; }
...
Hasher = new SHA1Managed();
...
Convert.ToBase64String(Hasher.ComputeHash(Encoding.ASCII.GetBytes(jsonString)));

To try and debug the problem I split it out like this:

Prehash = PreHashBuilder.ToString();
ASCIIBytes = Encoding.ASCII.GetBytes(Prehash);
HashedBytes = Hasher.ComputeHash(ASCIIBytes);
Hash = Convert.ToBase64String(HashedBytes);

I then added a route which spits out the above values and used Beyond Compare to compare the differences.

Byte arrays are converted to a string format for BeyondCompare use by using:

private static string GetString(byte[] bytes)
{
    StringBuilder sb = new StringBuilder();
    foreach (byte b in bytes)
    {
        sb.Append(b);
    }
    return sb.ToString();
} 

As you can see the byte array is displayed litterally as a sequence of bytes. It is not ‘converted’.

The Problem

I discovered that the Prehash and ASCIIBytes values were the same, but the HashedBytes values were different – which meant that the Hash was also different.

I restarted the IIS WebSites on the 4 server boxes several times and, when they had different hashes, compared the values in BeyondCompare. In everycase it was the “HashedBytes” value that was different (the results of SHA1Managed.ComputeHash(…))

The Question

What am I doing wrong? The input to the ComputeHash function is identical. Is SHA1Managed machine dependent? That doesn’t make since because half the time the 4 machines have the same hash.

I have searched StackOverFlow and Bing but have been unable to find anyone else with this problem. The closest thing I could find was people with problems with their encoding, but I think I have proven that the encoding is not an issue.

Output

I was hoping not to dump everything here because of how long it is, but here is a snipet of the dump I am comparing:

Hash:o1ZxBaVuU6OhE6De96wJXUvmz3M=
HashedBytes:163861135165110831631611916022224717299375230207115
ASCIIBytes:1151169710310146991111094779114100101114831011141181059910147115101114118105991014611511899591151051031101171129511510111411810599101114101102101114101110991011159598979910710111010011111410010111411510111411810599101951185095117114108611041161161125847471051159897991071011101004610910211598101115116971031014699111109477911410010111483101114118105991014711510111411810599101461151189947118505911510510311011711295115101114118105991011141011021011141011109910111595989799107101110100112971211091011101161151161111141011151011141….
Prehash:…

When I compare the two pages on the different servers the ASCII Bytes are identical but the HashedBytes are not. The dump method I use for the bytes does no conversions, it simply dumps each byte out in sequence. I could delimit the bytes with a ‘.’ I suppose.

Follow Up
I have made the b.ToString(CultureInfo.InvariantCulture) change and have made the HashAlgorithm a local variable instead of a static property. I am waiting for the code to deploy to the servers.

Answer 1

I have been trying to duplicate the issue but have been unable to do so once I made the SHA1Managed property a local variable instead of global static.

The problem was with Multi-Threading. My code was thread safe except for the SHA1Managed class that I had marked static. I assumed that SHA1Managed.ComputeHash would be thread safe underneath but apparently it is not if marked internal static.

To repeat, SHA1Managed.ComputeHash is not thread safe if marked internal static.

MSDN states:

Any public static (Shared in Visual Basic) members of this type are thread safe. Any instance members are not guaranteed to be thread safe.

I don’t know why internal static behaves differently than public static.

I would mark @pst as the answer and add a comment to clarify the problem, but @pst made a comment so I can’t mark it as the answer.

Thanks for all your input.