Question

I need to get the space-separated tokens in a string, but I also need to know the character position within the original string at which each token starts. Is there any way to do this with StringTokenizer. Also, as I understand it, this is a legacy class; is there a better alternative to using StringTokenizer.

Answer 1

You should always use String#split() to split your string rather than StringTokenizer.

However, since you also want the position of the tokens in your string, then it would be better to use Pattern and Matcher class. You have got Matcher#start() method which gives the position of the string matching the pattern.

Here’s an example: –

String str = "abc asf basdfasf asf";
Matcher matcher = Pattern.compile("\\S+").matcher(str);

while (matcher.find()) {
    System.out.println(matcher.start() + ":" + matcher.group());
}

The pattern \\S+ matches the non-space characters from that string. Using Matcher#find() methods returns all the matched substring.