Question

I’m trying to split a string into tokens (via regular expressions)
in the following way:

Example #1
input string: 'hello'
first token: '
second token: hello
third token: '

Example #2
input string: 'hello world'
first token: '
second token: hello world
third token: '

Example #3
input string: hello world
first token: hello
second token: world

i.e., only split up the string if it is NOT in single quotation marks, and single quotes should be in their own token.

This is what I have so far:

string pattern = @"'|\s";
Regex RE = new Regex(pattern);
string[] tokens = RE.Split("'hello world'");

This will work for example #1 and example #3 but it will NOT work for example #2.
I’m wondering if there’s theoretically a way to achieve what I want with regular expressions

Answer 1

While it would be possible to match ' and the text inside separately, and also alternatively match the text alone, RegExp does not allow an indefinite number of matches. Or better said, you can only match those objects you explicitely state in the expression. So ((\w+)+\b) could theoretically match all words one-by-one. The outer group will correctly match the whole text, and also the inner group will match the words separately correctly, but you will only be able to reference the last match.

There is no way to match a group of matched matches (weird sentence). The only possible way would be to match the string and then split it into separate words.