Question

It seems that something weird is happening in my code below. When any of the printf in the nthtoken() is commented, no token is displayed in main(), but when any of the nthtoken()’s printf is uncommented, token is displayed in main.

 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>

 char *nthtoken (char *origStr, char *delimiters, int nth);

 int main (void) {

      char *str = "in principio creavit deus caelum et terram";
      char *delims = " ";

      char *tok = nthtoken (str, delims, 3);
      printf ("token: %s\n", tok);

      return (EXIT_SUCCESS);
 }

 char *nthtoken (char *origStr, char *delimiters, int nth) {

      char str[strlen (origStr)];
      strncpy (str, origStr, strlen (origStr) + 1);

      char *token = NULL;
      token = strtok (str, delimiters);
      // printf ("first token: %s\n", token);

      int i = 0;
      for (i = 0; i < nth; i++) {
           token = strtok (NULL, delimiters);
           //printf ("token inside the loop: %s\n", token); 
      }
      // printf ("token before returning to main(): %s\n", token);
      return token;
 }

Answer 1

This is a classic mistake. token is local. Returning it would produce unpredictable result. You might get it in printf, if it hasn’t been over written, but it is not guaranteed and you should not do it. Bottom line is that upon function termination local variables are out of scope and technically no longer exist to compiler or run time.

Either malloc and copy the token to it (and free it in main) or pass it to the function as an out parameter, for instance

char * ret_string = malloc(MAX_SIZE); // ensure this is always sufficiently large
strcpy(ret_string, token);
return ret_string;

Don’f forget to free the malloc memory in main.

Edit: Thread safety note – As @grhegde suggested, Just for the record, strtok is not thread safe.If you are in a multithreaded environment, then you better find an alternative.

To further explain this, strtok uses a global buffer to keep track of tokens, this means that it cannot be called for tokenizing another string while one string is being tokenized
A standard solution to do this is often given with a function called strtok_r (r for re-entrant) where the implementation uses a local buffer for each call and hence can be called easily from multiple threads.