Question

Do I need to escape a single white space in a regular expression? I know it works either of the following ways in Python: re.compile(r'\s'), re.compile(r' ')and re.compile(r'\ '). So it seems that you can either escape it or not with Python re module.

But when I was trying to define a rule for a single whitespace via ply.lex, things seem to be a little different.

##########################
# r'\s' works with ply.lex
def t_WHITESPACE(token):
  r'\s'
  pass

##########################
# r'\ ' also works 
def t_WHITESPACE(token):
  r'\ '
  pass

##########################
# r' ' DOES NOT work 
# A SyntaxError was raised with the message "Regular expression for rule 't_WHITESPACE' matches empty string"
def t_WHITESPACE(token):
  r' '
  pass

Does ply.lex make any special processing with the rule’s doc string?

Answer 1

I don’t know ply, but this looks like it compiles the regex using the re.VERBOSE option which means you do need to escape literal whitespace or use \s.

Also, you need to escape hashes (#) or the following parts will be interpreted as comments.