Question

Hi I want to tokenize a string in C

Following is the string.

{Job Started}{Job Running}{Job Running}{Job Finished}

I want to tokenize on { and } , so that I get “Job Started”, “Job Running” and “Job Finished”

I also want same delimiter to be used as escaped characters

{Job Started}{Job \{ID1\} Running}{Job \{ID2\} Running}{Job Finished}

Should return me following

Job Started, Job {ID1} Running, Job {ID2} Running, Job Finishied.

I have solution with pointer airthmatic, but want to avoid re-iterating on the input string more than once.

Any suggestion.

Answer 1

You can use a simple finite state machine:

#include <stdio.h>

int main() {
    char *src = "{Job Started}{Job \\{ID1\\} Running}{Job \\{ID2\\} Running}{Job Finished}";

    char token[100] = {}, *dst = token, ch;

    int state = 0;
    while ((ch = *src++) != 0) {
        switch (state) {
            case 0:
                if (ch == '{') state = 1;
                break;
            case 1:
                switch (ch) {
                    case '}':
                        printf("token: %s\n", token);
                        dst = token;
                        *dst = 0;
                        state = 0;
                        break;
                    case '\\':
                        state = 2;
                        break;
                    default:
                        *dst++ = ch;
                        *dst = 0;
                }
                break;
            case 2:
                *dst++ = ch;
                *dst = 0;
                state = 1;
                break;
        }
    }
}