Question

I am using Scala combinatorial parser by extending scala.util.parsing.combinator.syntactical.StandardTokenParser. This class provides following methods

def ident : Parser[String] for parsing identifiers and

def numericLit : Parser[String] for parsing a number (decimal I suppose)

I am using scala.util.parsing.combinator.lexical.Scannersfrom scala.util.parsing.combinator.lexical.StdLexicalfor lexing.

My requirement is to parse a hexadecimal number (without the 0x prefix) which can be of any length. Basically a grammar like: ([0-9]|[a-f])+

I tried integrating Regex parser but there are type issues there. Other ways to extend the definition of lexer delimiter and grammar rules lead to token not found!

Answer 1

As I thought the problem can be solved by extending the behavior of Lexer and not the Parser. The standard lexer takes only decimal digits, so I created a new lexer:

class MyLexer extends StdLexical {
  override type Elem = Char
  override def digit = ( super.digit | hexDigit )
  lazy val hexDigits = Set[Char]() ++ "0123456789abcdefABCDEF".toArray
  lazy val hexDigit = elem("hex digit", hexDigits.contains(_))
}

And my parser (which has to be a StandardTokenParser) can be extended as follows:

object ParseAST extends StandardTokenParsers{

  override val lexical:MyLexer = new MyLexer()
  lexical.delimiters += ( "(" , ")" , "," , "@")
  ...
 }

The construction of the “number” from digits is taken care by StdLexical class:

class StdLexical {
...

def token: Parser[Token] = 
    ...
| digit~rep(digit)^^{case first ~ rest => NumericLit(first :: rest mkString "")}
}

Since StdLexical gives just the parsed number as a String it is not a problem for me, as I am not interested in numeric value either.