Question

I would like to tokenize the following string.
“I went home with Mark’s brother, to play a video game.”
result should be like this.

I
went
home
with
Mark
'
s
brother
to
,
play
a
video
game
.

Can you please tell me how to do it using regex.

Answer 1

If you really want the ,, . and ' as separate tokens, you could split like this:

String str = "I went home with Mark's brother, to play a video game.";
String[] tokens = str.split("(\\s|(?=[,.'])|(?<=[,.']))");