Question

I receive data as a vector<char>, from which I need to create a string. Vector may contain utf-16 characters (i.e. null bytes) and is a fixed size. Actual data is padded with null bytes to this fixed sized. So, for example, I can have the following vector:

\0 a \0 b \0 c \0 d \0 \0 \0 \0

Fixed size is 12 and the vector contains utf-16 string “abcd” padded with 4 null chars to size.

From this, I need to actually extract this string. I already have the code for converting from utf-16 to string, the thing where I got myself confused is find the number of characters (bytes) in the vector without the padding. In the example above, the number is 8.

I started by doing something like:

std::string CrmxFile::StringFromBytes(std::vector<char> data, int fixedsize) {

    std::vector<char>iterator it = data.rbegin();
    while(it != data.rend() && *it == '\0') {
        it++;
    }

    return std::string(&data[0], fixedsize - (it - data.rbegin());
}

However in the full context, the vector contains a lot of data and I need to do the above manipulation with only a specified part of it. For example, the vector may contain 1000 elements and I need to get the string that starts at position 30 and goes for a max of 12 chars. Of course, I can create another vector and copy the required 21 characters into it before applying the above logic, but I feel that I should be able to do something directly on the given vector. Yet, I can’t grasp what iterators I am comparing with what. Any help is appreciated.

Answer 1

Now, this is embarrassing: vector<char>::iterator is obviously a random access iterator, therefore I can decrement it. Hence my method now looks like this:

std::string CrmxFile::StringFromBytes(std::vector<char> data, int fixedsize) {
    std::vector<char>::iterator begin = data.begin() + start;
    std::vector<char>::iterator end = start + length - 1;
    while(it >= begin  && *it == '\0') {
        it--;
    }

    if(it >= begin) {
        int len = it - begin + 1;
        if(IsUtf8Heuristic(begin, begin + len) {
            return std::string(begin, begin + len);
        }
        else {  //(heuristically this is utf-16)
            len = ((len + 1) >> 1) << 1;
            std::string res;
            ConvertUtf16To8(begin, begin + len, std::back_inserter(res));
            return res;
        }
    }
    else {
        return "";
    }
}