Question

I’ve got a function that splits up a string into various sections and then parses them, but when converting a string to char* I get a malformed output.

int parseJob(char * buffer)
{ // Parse raw data, should return individual jobs
    const char* p;
    int rows = 0;
    for (p = strtok( buffer, "~" );  p;  p = strtok( NULL, "~" )) { 
        string jobR(p);
        char* job = &jobR[0];
        parseJobParameters(job); // At this point, the data is still in good condition
    }
    return (1);
}

int parseJobParameters(char * buffer)
{ // Parse raw data, should return individual job parameters
    const char* p;
    int rows = 0;
    for (p = strtok( buffer, "|" );  p;  p = strtok( NULL, "|" )) { cout<<p; } // At this point, the data is malformed.
    return (1);
}

I don’t know what happens between the first function calling the second one, but it malforms the data.

As you can see from the code example given, the same method to convert string to char* is used and it works fine.

I’m using Visual Studio 2012/C++, any guidance and code examples will be greatly appreciated.

Answer 1

The “physical” reason your code does not work has nothing to do with std::string or C++. It wouldn’t work in pure C as well. strtok is a function that stores its intermediate parsing state in some global variable. This immediately means that you cannot use strtok to parse more than one string at a time. Starting the second parse session before finishing the first would override the internal data stored by the first parse session, thus ruining it beyond repair. In other words, strtok parse sessions must not overlap. In your code they do overlap.

---

Also, in C++03 the idea of using std::string with strtok directly is doomed from the start. The internal sequence stored in std::string is not guaranteed to be null-terminated. This means that generally &jobR[0] is not a C-string. It can’t be used with strtok. To convert a std::string to a C-string you have to use c_str(). But C-string returned by c_str() is non-modifiable.

In C++11 the null-termination is supposed to be visible through the [] operator, but still there seems to be no requirement to store the terminator object contiguously with the actual string, so &jobR[0] is still not a C-string even in C++11. C-string returned by c_str() or data() is non-modifiable.