Question

using only regular expression methods, the method String.replaceAll and ArrayList
how can i split a String into tokens, but ignore delimiters that exist inside quotes?
the delimiter is any character that is not alphanumeric or quoted text

for example:
The string :

hello^world’this*has two tokens’

should output:

• hello
• worldthis*has two tokens

Answer 1

Use a Matcher to identify the parts you want to keep, rather than the parts you want to split on:

String s = "hello^world'this*has two tokens'";
Pattern pattern = Pattern.compile("([a-zA-Z0-9]+|'[^']*')+");
Matcher matcher = pattern.matcher(s);
while (matcher.find()) {
    System.out.println(matcher.group(0));
}

See it working online: ideone