I am currently working on a network tool that needs to decode/encode a particular protocol that packs fields into dense bit arrays at arbitrary positions. For example, one part of the protocol uses 3 bytes to represent a number of different fields:
Bit Position(s) Length (In Bits) Type
0 1 bool
1-5 5 int
6-13 8 int
14-22 9 uint
23 1 bool
As you can see, several of the fields span multiple bytes. Many (most) are also shorter than the built-in type that might be used to represent them, such as the first int field which is only 5 bits long. In these cases, the most significant bits of the target type (such as an Int32 or Int16) should be padded with 0 to make up the difference.
My problem is that I am having a difficult time processing this kind of data. Specifically, I am having a hard time figuring out how to efficiently get arbitrary length bit arrays, populate them with the appropriate bits from the source buffer, pad them to match the target type, and convert the padded bit arrays to the target type. In an ideal world, I would be able to take the byte[3] in the example above and call a method like GetInt32(byte[] bytes, int startBit, int length).
The closest thing in the wild that I’ve found is a BitStream class, but it appears to want individual values to line up on byte/word boundaries (and the half-streaming/half-indexed access convention of the class makes it a little confusing).
My own first attempt was to use the BitArray class, but that proved somewhat unwieldy. It’s easy enough to stuff all the bits from the buffer into a large BitArray, transfer only the ones you want from the source BitArray to a new temporary BitArray, and then convert that into the target value…but it seems wrong, and very time consuming.
I am now considering a class like the following that references (or creates) a source/target byte[] buffer along with an offset and provides get and set methods for certain target types. The tricky part is that getting/setting values may span multiple bytes.
class BitField
{
private readonly byte[] _bytes;
private readonly int _offset;
public BitField(byte[] bytes)
: this(bytes, 0)
{
}
public BitField(byte[] bytes, int offset)
{
_bytes = bytes;
_offset = offset;
}
public BitField(int size)
: this(new byte[size], 0)
{
}
public bool this[int bit]
{
get { return IsSet(bit); }
set { if (value) Set(bit); else Clear(bit); }
}
public bool IsSet(int bit)
{
return (_bytes[_offset + (bit / 8)] & (1 << (bit % 8))) != 0;
}
public void Set(int bit)
{
_bytes[_offset + (bit / 8)] |= unchecked((byte)(1 << (bit % 8)));
}
public void Clear(int bit)
{
_bytes[_offset + (bit / 8)] &= unchecked((byte)~(1 << (bit % 8)));
}
//startIndex = the index of the bit at which to start fetching the value
//length = the number of bits to include - may be less than 32 in which case
//the most significant bits of the target type should be padded with 0
public int GetInt32(int startIndex, int length)
{
//NEED CODE HERE
}
//startIndex = the index of the bit at which to start storing the value
//length = the number of bits to use, if less than the number of bits required
//for the source type, precision may be lost
//value = the value to store
public void SetValue(int startIndex, int length, int value)
{
//NEED CODE HERE
}
//Other Get.../Set... methods go here
}
I am looking for any guidance in this area such as third-party libraries, algorithms for getting/setting values at arbitrary bit positions that span multiple bytes, feedback on my approach, etc. I included the class above for clarification and am not necessarily looking for code to fill it in (though I won’t argue if someone wants to work it out!).
As promised, here is the class I ended up creating for this purpose. It will wrap an arbitrary byte array at an optionally specified index and allowing reading/writing at the bit level. It provides methods for reading/writing arbitrary blocks of bits from other byte arrays or for reading/writing primitive values with user-defined offsets and lengths. It works very well for my situation and solves the exact question I asked above. However, it does have a couple shortcomings. The first is that it is obviously not greatly documented – I just haven’t had the time. The second is that there are no bounds or other checks. It also currently requires the MiscUtil library to provide endian conversion. All that said, hopefully this can help solve or serve as a starting point for someone else with a similar use case.