Question

I am working on a display/control utility to replace an ancient dedicated hardware controller for a piece of industrial machinary. The controller itself is beyond repair (someone replaced the 1 amp fuse with a 13 amp one “because it kept blowing”). The hardware interface is through a standard RS232 port. The data format is dedicated:

No control characters are used with the exeption of ETB (Chr 23) to demark end of a message.

The data is 7-bit, but only a subset of the possible 7-bit characters is used. The content of each 7-bit data character is therefore effectively reduced to only 6 bits of data.

The data is not character aligned, e.g. for the first message type, the first 3 bits are the message type, the next 8 bits are a counter, the next 15 bits are a data value, next 7 bits are a value etc

So reducing the data from it’s 7-bit carrier to it’s 6 bit content gives (for example)

|  6 ||  6 ||  6 ||  6 ||  6 ||  6 ||  6 ||~ 
001001010001010100101010101101010101110111 ~
|3||  8   ||     15       ||  7   || ~~   
 M    C          D            D        D
 s    o          a            a        a 
 g    u          t            t        t 
      n          a            a        a
      t 

Specific messages are fixed length but the different messages are of different lengths and contain different numbers of parameters.

I have a working prototype handling one specific message type, but it is currently using way too many case statements ;-).

I am looking for suggestions as to a clean way of handling such packed, arbitrary bit length data?

Answer 1

Use SHL/SHR along with masking to read out of your buffer. I would write a few functions to operate against the buffer (which I would declare as an array of byte) and return the value of a specific number of bits form a starting bit position. For instance, lets say that your largest value will never be more than 16 bits (a word). Since your mapped to an array of bytes, if you always grab 3 bytes from the array (watch for the upper bounds) and throw into an integer you can then mask and shift to get your specific value. The location of the bytes you want to get will then be (assuming a 0 based array):

Byte1Index = FirstBit DIV 8;
Byte2Index = Byte1Index + 1;
Byte3Index = Byte1Index + 2;

Pull these into your integer from your array.

TempInt := 0;
if Byte1Index <= High(Buffer) then
  TempInt := Buffer[Byte1Index];
if Byte2Index <= High(Buffer) then 
  TempInt := TempInt OR ((Buffer[Byte2Index] and $000000FF) SHL 8);
if Byte3Index <= High(Buffer) then
  TempInt := TempInt OR ((Buffer[Byte2Index] and $000000FF) SHL 16);

Then adjust your result to align properly

if FirstBit MOD 8 <> 0 then
  TempInt := TempInt SHR (FirstBit MOD 8);

Then mask against the most number of bits you want to return:

Result := TempInt AND $00003FFF;

You can build the final mask programatically:

FinalMask := ($FFFFFFFF shl bitcount) xor $FFFFFFFF;

EDIT

This method is currently limited to 32 bits, but can be extended to at the most 64 bits by changing the masks from 32bit integers to 64bit integers (from $FFFFFFFF to $FFFFFFFFFFFFFFFF for example). Performance gains can also be made by not loading the extra bytes if they aren’t needed, I just included here examples for 16 bits, however you will always want to grab at least an extra byte if the Bitsneeded + FirstBiT MOD 8 <> 0

For the messaging side of things I would create a base object which knows how to read data out of a buffer, then extend this into an object which knows how to also read parameters, and then create object descendants which know how to process each message type. You would still have a case statement, but it would be at a simple dispatcher level doing nothing more than passing the buffer off to the appropriate object to handle.