Question

I have written my own SHA1 implementation in MATLAB, and it gives correct hashes. However, it’s very slow (a string a 1000 a‘s takes 9.9 seconds on my Core i7-2760QM), and I think the slowness is a result of how MATLAB implements bitwise logical operations (bitand, bitor, bitxor, bitcmp) and bitwise shifts (bitshift, bitrol, bitror) of integers.

Especially I wonder the need to construct fixed-point numeric objects for bitrol and bitror using fi command, because anyway in Intel x86 assembly there’s rol and ror both for registers and memory addresses of all sizes. However, bitshift is quite fast (it doesn’t need any fixed-point numeric costructs, a regular uint64 variable works fine), which makes the situation stranger: why in MATLAB bitrol and bitror need fixed-point numeric objects constructed with fi, whereas bitshift does not, when in assembly level it all comes down to shl, shr, rol and ror?

So, before writing this function in C/C++ as a .mex file, I’d be happy to know if there is any way to improve the performance of this function. I know there are some specific optimizations for SHA1, but that’s not the issue, if the very basic implementation of bitwise rotations is so slow.

Testing a little bit with tic and toc, it’s evident that what makes it slow are the loops in with bitrol and fi. There are two such loops:

%# Define some variables.
FFFFFFFF = uint64(hex2dec('FFFFFFFF'));

%# constants: K(1), K(2), K(3), K(4).
K(1) = uint64(hex2dec('5A827999'));
K(2) = uint64(hex2dec('6ED9EBA1'));
K(3) = uint64(hex2dec('8F1BBCDC'));
K(4) = uint64(hex2dec('CA62C1D6'));

W = uint64(zeros(1, 80));

... some other code here ...

%# First slow loop begins here.

for index = 17:80
    W(index) = uint64(bitrol(fi(bitxor(bitxor(bitxor(W(index-3), W(index-8)), W(index-14)), W(index-16)), 0, 32, 0), 1));
end

%# First slow loop ends here.

H = sha1_handle_block_struct.H;

A = H(1);
B = H(2);
C = H(3);
D = H(4);
E = H(5);

%# Second slow loop begins here.

for index = 1:80
    rotatedA = uint64(bitrol(fi(A, 0, 32, 0), 5));

    if (index <= 20)
        % alternative #1.
        xorPart = bitxor(D, (bitand(B, (bitxor(C, D)))));
        xorPart = bitand(xorPart, FFFFFFFF);
        temp = rotatedA + xorPart + E + W(index) + K(1);
    elseif ((index >= 21) && (index <= 40))
        % FIPS.
        xorPart = bitxor(bitxor(B, C), D);
        xorPart = bitand(xorPart, FFFFFFFF);
        temp = rotatedA + xorPart + E + W(index) + K(2);
    elseif ((index >= 41) && (index <= 60))
        % alternative #2.
        xorPart = bitor(bitand(B, C), bitand(D, bitxor(B, C)));
        xorPart = bitand(xorPart, FFFFFFFF);
        temp = rotatedA + xorPart + E + W(index) + K(3);
    elseif ((index >= 61) && (index <= 80))
        % FIPS.
        xorPart = bitxor(bitxor(B, C), D);
        xorPart = bitand(xorPart, FFFFFFFF);
        temp = rotatedA + xorPart + E + W(index) + K(4);
    else
        error('error in the code of sha1_handle_block.m!');
    end

temp = bitand(temp, FFFFFFFF);
E = D;
D = C;
C = uint64(bitrol(fi(B, 0, 32, 0), 30));
B = A;
A = temp;
end

%# Second slow loop ends here.

Measuring with tic and toc, the entire computation of SHA1 hash of message abc takes on my laptop around 0.63 seconds, of which around 0.23 seconds is passed in the first slow loop and around 0.38 seconds in the second slow loop. So is there some way to optimize those loops in MATLAB before writing a .mex file?

Answer 1

There’s this DataHash from the MATLAB File Exchange that calculates SHA-1 hashes lightning fast.
I ran the following code:

x = 'The quick brown fox jumped over the lazy dog';  %# Just a short sentence
y = repmat('a', [1, 1e6]);                           %# A million a's
opt = struct('Method', 'SHA-1', 'Format', 'HEX', 'Input', 'bin');
tic, x_hashed = DataHash(uint8(x), opt), toc
tic, y_hashed = DataHash(uint8(y), opt), toc

and got the following results:

x_hashed = F6513640F3045E9768B239785625CAA6A2588842
Elapsed time is 0.029250 seconds.

y_hashed = 34AA973CD4C4DAA4F61EEB2BDBAD27316534016F
Elapsed time is 0.020595 seconds.

I verified the results with a random online SHA-1 tool, and the calculation was indeed correct. Also, the 10⁶ a’s were hashed ~1.5 times faster than the first sentence.

So how does DataHash do it so fast??? Using the java.security.MessageDigest library, no less!
If you’re interested with a fast MATLAB-friendly SHA-1 function, this is the way to go.

However, if this is just an exercise for implementing fast bit-level operations, then MATLAB doesn’t really handle them efficiently, and in most cases you’ll have to resort to MEX.