Question

hi im doing string tokenization similarly to the example below. however, within the while loop, i will be changing letter ‘a’ to ‘hellow’ for example. im getting segmentation fault when trying to change pch before assigning to myVar[i]. how should i go about doing it?

    map <int, char*> myVar;
    char str[] ="- This, a sample string.";
    char * pch;
    printf ("Splitting string \"%s\" into tokens:\n",str);
    pch = strtok (str," ,.-");
    int i = 0;

    while (pch != NULL)
    {
        printf ("%s\n",pch);

        //modify token value
        stringstream strStream;
        strStream << "hello_world";

        char newStr[7] = {0};
        memcpy(newStr, strStream, 7);

        myVar[i] = (char*)newStr;
        pch = strtok (NULL, " ,.-");
        i++;
     }

Answer 1

I see two bugs inside of your while loop:

1) you are passing the stringstream itself, not the data it contains, to memcpy(). You are relying on the stringstream::operator void*() conversion operator. You are not supposed to deference that pointer, as it does not point at actual data. It is merely a flag to indicate whether the stringstream is valid or not. To pass the stringstream data to memcpy(), you have to call its str() method first to get a std::string containing the data, and then call its c_str() method to pass that data to memcpy().

2) when you are inserting values into your std::map, you are inserting a local char[] variable each time. That char[] goes out of scope immediately afterwards, leaving the std::map containing pointers to random locations on the stack. Given the code you have shown, the char[] buffers are likely to reuse the same stack space each time.

Since you are using C++, you really should be using more C++-oriented things, like std::string, std::cout, etc.

Try this:

std::map <int, std::string> myVar;
std::string str = "- This, a sample string.";
std::cout << "Splitting string \"" << str << "\" into tokens:" << std::endl;
size_t start = 0;
int i = 0;

do
{
    std::string token;

    size_t pos = str.find_first_of(" ,.-", start);
    if (pos != std::string::npos)
    { 
        token = str.substr(start, pos-start);
        start = pos + 1;
    }
    else
    {
        token = str.substr(start);
        start = std::string::npos;
    }

    std::cout << token << std::endl;

    //modify token value
    myVar[i] = "hello_world";

    ++i;
 }
 while (start != std::string::npos);