Question

I have a large string accepted from TCP listner which is in following format

“1,7620257787,0123456789,99,0922337203,9223372036,32.5455,87,12.7857,1/1/2012,9223372036:1,7620257787,0123456789,99,0922337203,9223372036,32.5455,87,12.7857:2/1/2012,234234234:3,7620257787,01234343456789,99,0922337203,9223372036,32.5455,87,12.7857,1/1/2012,9223372036:34,76202343457787,012434343456789,93339,34340922337203,9223372036,32.5455,87,12.7857,1/1/2012,9223372036”

You can see that this is a : seperated string which contains Records which are comma seperated fields.

I am looking for the best (fastest) way that split the string in given number of chunks and take care that one chunk should contain full record (string upto ‘:’)

or other way of saying , there should not be any chunck which is not ending with :

e.g. 20 MB string to 4 chunks of 5 MB each with proper records (thus size of each chunk may not be exactly 5 MB but very near to it and total of all 4 chunks will be 20 MB)

I hope you can understand my question (sorry for the bad english)

I like the following link , but it does not take care of full record while spliting also don’t know if that is the best and fastest way.

Split String into smaller Strings by length variable

Answer 1

The idea is to divide the lenght of your data for the num of blocks required, then look backwards to search the last sep in the current block.

    private string[] splitToBlocks(string data, int numBlocks, char sep)
    {
        // We return an array of the request length
        if (numBlocks <= 1 || data.Length == 0)
        {
            return new string [] { data };
        }

        string[] result = new string[numBlocks];

        // The optimal size of each block
        int blockLen = (data.Length / numBlocks);

        int idx = 0; int pos = 0; int lastSepPos = blockLen;
        while (idx < numBlocks)
        {
            // Search backwards for the first sep starting from the lastSepPos
            char c = data[lastSepPos];
            while (c != sep) { lastSepPos--; c = data[lastSepPos]; }

            // Get the block data in the result array
            result[idx] = data.Substring(pos, (lastSepPos + 1) - pos);

            // Reposition for then next block
            idx++;
            pos = lastSepPos + 1;

            if(idx == numBlocks-1)
                lastSepPos = data.Length - 1;
            else
                lastSepPos = blockLen * (idx + 1);
        }
        return result;
    }

Please test it. I have not fully tested for fringe cases.