Question

i am looking for a method in java, to extract certain bytes from an input stream.
for example
i will have this stream of data occuring

0x01,0x02,0x00,0x01,0x00,0x01,0x03,0x04,0x00,0x01,0x09,0x08,0x00,0x01,0x00,0x01

my encoding scheme is type data ending
firstly i will check the first byte,
then i will want to store all the data in a byte array from 0x01 untill the occurance of 0x00,0x01,0x00,0x01 except for the 0x01‘s

so the first piece of data i would place into the array

0x01,0x02,0x00,0x00 

and then onto the next ,
this begins with a 0x03 and ends with 0x00,0x01,0x00,0x01
i would like for this to be placed in another byte array as,

0x03,0x04,0x00,0x01,0x09,0x08,0x00,0x00

how would i go about doing this, i began with using

a ByteArrayOutputStream to add dynamically to the byte array, without needing to know the size,
but im lost on the logic on how would extract out each pattern and remove each 0x01 following a 0x00,
also im rading a byte in from an input stream , one byte at a time (its the only way i can get the bytes)

Answer 1

You need a finite-state recognizer. For your simple syntax the following pseudocode should do the trick:

state = 0;
while( (byte=input.read()) != EOF)
{
    switch(state)
    {
        case 0:     // "normal" state
            if (byte == 0x00)
            {
                state = 1;
                buf.append(byte);
            }
            else
                output.write(byte)
            break;
        case 1:     // We've seen a 0x00
            if (byte == 0x00)
            {
                state = 1;
                output.write(buf);
            }
            else if (byte == 0x01)
            {
                state = 2;
                buf.append(byte);
            }
            else
            {
                output.write(buf);
                buf.clear();
                state = 0;
            }
            break;
        case 2:     // We've seen 0x00,0x01
            if (byte == 0x00)
            {
                state = 3;
                buf.append(byte);
            }
            else if (byte == 0x01)
            {
                output.write(0x00);
                buf.clear();
                state = 0;
            }
            else
            {
                output.write(buf);
                buf.clear();
                state = 0;
            }
            break;
        case 3:     // We've seen 0x00,0x01,0x00
            if (byte == 0x00)
            {
                state = 1;
                output.write(buf);
                buf.clear();
                buf.append(byte);
            }
            else if (byte == 0x01)
            {
                // The last four input bytes were 0x00,0x01,0x00,0x01
                state = 0;
                output.write(0x00,0x00);
                buf.clear
            }
            else
            {
                output.write(buf);
                buf.clear();
                state = 0;
            }
            break;
    }
}
if (!buf.empty()) output.write(buf);

This works by reading bytes one at a time.

If it detects a 0x00, we need to start looking for the delimiter pattern but save the bytes in case later we find it was a false alarm. The “state” variable keeps track of what we’ve seen so far. At each point if the input matches the next expected delimiter byte we save it, bump the state and keep going. If at any point we don’t get the next expected delimiter byte we just write out all the saved data, clear the buffer and go back to “normal” state. However, if we eventually see the entire delimiter string, we write out 0x00,0x00 and discard the saved bytes (which would be 0x00,0x01,0x00,0x01).

EDIT: Code modified to handle additional condition from OP and comment from @Shaded