Question

I’m trying to tokenize a sentence into words separated only by single space.
I need to get words from the sentence and then write them onto subscribed array.
Here is my work:

void writeToStrArrayOneByOne(char words[10][20], char *sentence){

       char *tokenPtr;
       int j = 0;
       int a,i;
       tokenPtr = strtok(sentence," ");

       while(tokenPtr != NULL){


          a = strlen(tokenPtr);

          for(i=0;i<a;i++){

                words[j][i] = tokenPtr[i];
          }
          tokenPtr = strtok(NULL," ");
          j++;

       }

And I call the function with only this in order to debug from main :

     char words[10][20];
     char *sentence = "this is a token";
     writeToStrArrayOneByOne(words,sentence);

However it freezes like when it does when it’s EOF exception.
Appreciate any help, thanks.

Answer 1

strtok() modifies the string it’s passed. You’re giving it a string literal which cannot be modified.

Try:

char sentence[] = "this is a token";

Which will make sentence a modifiable array of characters.

Also keep in mind that the loop where you copy the tokens won’t put a null terminator at the end of each entry – I suspect that you’ll probably want those. Maybe try:

strcpy(words[j], tokenPtr);

Other things you should think about include:

• handling tokens that might be too long
• getting a string with too many tokens
• what needs to be done if there is more than one space between tokens – strtok() doesn’t deal with that if it matters to you (I mention this only because in your question you specifically mention that tokens are “separated only by a single space”)