Question

I thought this will be trivial but I can’t get this to work.

Assume a line in a CSV file:
"Barack Obama", 48, "President", "1600 Penn Ave, Washington DC"

string[] tokens = line.split(',')

I expect this:

 "Barack Obama"
 48
 "President"
 "1600 Penn Ave, Washington DC"

but the last token is
'Washington DC' not
"1600 Penn Ave, Washington DC".

Is there an easy way to get the split function to ignore the comma within quotes?

I have no control over the CSV file and it doesn;t get sent to me. Customer A will be using the app to read files provided by an external individual.

Answer 1

You might have to write your own split function.

• Iterate through each char in the string
• When you hit a " character, toggle a boolean
• When you hit a comma, if the bool is true, ignore it, else, you have your token

Here’s an example:

public static class StringExtensions
{
    public static string[] SplitQuoted(this string input, char separator, char quotechar)
    {
        List<string> tokens = new List<string>();

        StringBuilder sb = new StringBuilder();
        bool escaped = false;
        foreach (char c in input)
        {
            if (c.Equals(separator) && !escaped)
            {
                // we have a token
                tokens.Add(sb.ToString().Trim());
                sb.Clear();
            }
            else if (c.Equals(separator) && escaped)
            {
                // ignore but add to string
                sb.Append(c);
            }
            else if (c.Equals(quotechar))
            {
                escaped = !escaped;
                sb.Append(c);
            }
            else
            {
                sb.Append(c);
            }
        }
        tokens.Add(sb.ToString().Trim());

        return tokens.ToArray();
    }
}

Then just call:

string[] tokens = line.SplitQuoted(',','\"');

---

Benchmarks

Results of benchmarking my code and Dan Tao’s code are below. I’m happy to benchmark any other solutions if people want them?

Code:

string input = "\"Barak Obama\", 48, \"President\", \"1600 Penn Ave, Washington DC\""; // Console.ReadLine()
string[] tokens = null;

// run tests
DateTime start = DateTime.Now;
for (int i = 0; i < 1000000; i++)
    tokens = input.SplitWithQualifier(',', '\"', false);
Console.WriteLine("1,000,000 x SplitWithQualifier = {0}ms", DateTime.Now.Subtract(start).TotalMilliseconds);

start = DateTime.Now;
for (int i = 0; i<1000000;i++)
    tokens = input.SplitQuoted(',', '\"');
Console.WriteLine("1,000,000 x SplitQuoted =        {0}ms", DateTime.Now.Subtract(start).TotalMilliseconds);

Output:

1,000,000 x SplitWithQualifier = 8156.25ms
1,000,000 x SplitQuoted =        2406.25ms