Home/ Questions/Q 814517
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-15T01:32:05+00:00

I’m developing a syntax analyzer by hand in Java, and I’d like to use

  • 0

I’m developing a syntax analyzer by hand in Java, and I’d like to use regex’s to parse the various token types. The problem is that I’d also like to be able to accurately report the current line number, if the input doesn’t conform to the syntax.

Long story short, I’ve run into a problem when I try to actually match a newline with the Scanner class. To be specific, when I try to match a newline with a pattern using the Scanner class, it fails. Almost always. But when I perform the same matching using a Matcher and the same source string, it retrieves the newline exactly as you’d expect it too. Is there a reason for this, that I can’t seem to discover, or is this a bug, as I suspect?

FYI: I was unable to find a bug in the Sun database that describes this issue, so if it is a bug, it hasn’t been reported.

Example Code:

Pattern newLinePattern = Pattern.compile("(\\r\\n?|\\n)", Pattern.MULTILINE);
String sourceString = "\r\n\n\r\r\n\n";
Scanner scan = new Scanner(sourceString);
scan.useDelimiter("");
int count = 0;
while (scan.hasNext(newLinePattern)) {
    scan.next(newLinePattern);
    count++;
}
System.out.println("found "+count+" newlines"); // finds 7 newlines
Matcher match = newLinePattern.matcher(sourceString);
count = 0;
while (match.find()) {
    count++;
}
System.out.println("found "+count+" newlines"); // finds 5 newlines
Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Your useDelimiter() and next() combo is faulty. useDelimiter("") will return 1-length substring on next(), because an empty string does in fact sit between every two characters.

    That is, because "\r\n".equals("\r" + "" + "\n") so "\r\n" are in fact two tokens, "\r" and "\n", delimited by "".

    To get the Matcher-behavior, you need findWithinHorizon, which ignores delimiters.

        Pattern newLinePattern = Pattern.compile("(\\r\\n?|\\n)", Pattern.MULTILINE);
    String sourceString = "\r\n\n\r\r\n\n";
    Scanner scan = new Scanner(sourceString);
    int count = 0;
    while (scan.findWithinHorizon(newLinePattern, 0) != null) {
        count++;
    }
    System.out.println("found "+count+" newlines"); // finds 5 newlines

    API links

    • findWithinHorizon(Pattern pattern, int horizon)

      Attempts to find the next occurrence of the specified pattern […] ignoring delimiters […] If no such pattern is detected then the null is returned […] If horizon is 0, then […] this method continues to search through the input looking for the specified pattern without bound.

    Related questions

    • 0