Question

I have some code that tokenizes a equation input into a string array:

string infix = "( 5 + 2 ) * 3 + 4";
string[] tokens = tokenizer(infix, @"([\+\-\*\(\)\^\\])");
foreach (string s in tokens)
{
   Console.WriteLine(s);
}

Now here is the tokenizer function:

public string[] tokenizer(string input, string splitExp)
        {
            string noWSpaceInput = Regex.Replace(input, @"\s", "");
            Console.WriteLine(noWSpaceInput);
            Regex RE = new Regex(splitExp);
            return (RE.Split(noWSpaceInput));
        }

When I run this, I get all characters split, but there is an empty string inserted before the parenthesis chracters…how do I remove this?

//empty string here

(

5

+

2

//empty string here

)

*

3

+

4

Answer 1

I would just filter them out:

public string[] tokenizer(string input, string splitExp)
{
    string noWSpaceInput = Regex.Replace(input, @"\s", "");
    Console.WriteLine(noWSpaceInput);
    Regex RE = new Regex(splitExp);
    return (RE.Split(noWSpaceInput)).Where(x => !string.IsNullOrEmpty(x)).ToArray();
}