Question

I just wrote a program that tokenizes a char array using pointers. The program only needed to work with a space as the delimiter character. I just turned it in and got full credit, but after turning it in, I realized that this program worked only if the delimiter character was a space.

My question is, how could I make this program work with an arbitrary delimiter character?

The function I’ve shown you below returns a pointer to the next word in the char array. This is what I believe I need to change for it to work with any delimiter character.

Thanks!

Code:

char* StringTokenizer::Next(void) {
    pNextWord = pStart;

    if (*pStart == '\0') { return NULL; }

    while (*pStart != delim) {
        pStart++;
    }

    if (*pStart == '\0') { return NULL; }

    *pStart = '\0';
    pStart++;

    return pNextWord;
}

The printing loop in main():

while ((nextWord = tk.Next()) != NULL) {
    cout << nextWord << endl;
}

Answer 1

The simpliest way is to change your

while (*pStart != delim)

to something like

while (*pStart != ' ' && *pStart != '\n' && *pStart != '\t')

Or, you could make delim a string, and create a function that checks if a char is in the string:

bool isDelim(char c, const char *delim) {
   while (*delim) {
      if (*delim == c)
         return true;
      delim++;
   }
   return false;
}

while ( !isDelim(*pStart, " \n\t") ) 

Or, perhaps the best solution is to use one of the prebuilt functions for doing all this, such as strtok.