Question

Ok, so I understand that strtok modifies its input argument, but in this case, it’s collapsing down the input string into only the first token. Why is this happening, and what can I do to fix it? (Please note, I’m not talking about the variable “temp”, which should be the first token, but rather the variable “input”, which after one call to strtok becomes “this”)

#include <string.h>
#include <stdlib.h>
#include <stdio.h>

int main(int argc, char* argv[]) {
   char input[]="this is a test of the tokenizor seven";
   char * temp;
   temp=strtok(input," ");
   printf("input: %s\n", input); //input is now just "this"
}

Answer 1

When strtok() finds a token, it changes the character immediately after the token into a \0, and then returns a pointer to the token. The next time you call it with a NULL argument, it starts looking after the separators that terminated the first token — i.e., after the \0, and possibly further along.

Now, the original pointer to the beginning of the string still points to the beginning of the string, but the first token is now \0-terminated — i.e., printf() thinks the end of the token is the end of the string. The rest of the data is still there, but that \0 stops printf() from showing it. If you used a for-loop to walk over the original input string up to the original number of characters, you’d find the data is all still there.