Question

I’m a little surprised that there isn’t some information on this on the web, and I keep finding that the problem is a little stickier than I thought.

Here’s the rules:

1. You are starting with delimited/escaped data to split into an array.
2. The delimiter is one arbitrary character
3. The escape character is one arbitrary character
4. Both the delimiter and the escape character could occur in data
5. Regex is fine, but a good-performance solution is best
6. Edit: Empty elements (including leading or ending delimiters) can be ignored

The code signature (in C# would be, basically)

public static string[] smartSplit(                          string delimitedData,                           char delimiter,                           char escape) {} 

The stickiest part of the problem is the escaped consecutive escape character case, of course, since (calling / the escape character and , the delimiter): ////////, = ////,

Am I missing somewhere this is handled on the web or in another SO question? If not, put your big brains to work… I think this problem is something that would be nice to have on SO for the public good. I’m working on it myself, but don’t have a good solution yet.

Answer 1

void smartSplit(string const& text, char delim, char esc, vector<string>& tokens) {     enum State { NORMAL, IN_ESC };     State state = NORMAL;     string frag;      for (size_t i = 0; i<text.length(); ++i)     {         char c = text[i];         switch (state)         {         case NORMAL:             if (c == delim)             {                 if (!frag.empty())                     tokens.push_back(frag);                 frag.clear();             }             else if (c == esc)                 state = IN_ESC;             else                 frag.append(1, c);             break;         case IN_ESC:             frag.append(1, c);             state = NORMAL;             break;         }     }     if (!frag.empty())         tokens.push_back(frag); }