Question

I am trying to do a lexer for a subset of Java with JavaCC. And a variable name can be any combination of letter, digit and _, beginning with a letter. I have only one problem, reserved words (such as int, new, …) can not be used as a variable name and I was wondering how to declare this. Right now I have this where the reserved words are declared first, and then the rule for variable names, is it enought and then it will be to the parser to deal with it ?

//Reserved words
TOKEN:{
  < TOK_BOOLEAN : "boolean" > |
  < TOK_BREAK : "break" > |
  < TOK_CLASS : "class" >
}

TOKEN:{
  < TOK_ID : <LETTER> (<LETTER>|<DIGIT>|"_")+ > |
  < #DIGIT : ["0"-"9"] > |
  < #LETTER : ["a"-"z"] | ["A"-"Z"] >
}

TOK_ID is the rule for variable name.

Thank you and ask me if something is not clear.

Answer 1

The JavaCC lexer chooses the first definition that gains the longest match, so your definition should be sufficient.

This behavior is documented in the JavaCC TokenManager Tutorial. The JavaCC FAQ explains it here.