Question

I want to split an arithmetic expression into tokens, to convert it into RPN.

Java has the StringTokenizer, which can optionally keep the delimiters. That way, I could use the operators as delimiters. Unfortunately, I need to do this in PHP, which has strtok, but that throws away the delimiters, so I need to brew something myself.

This sounds like a classic textbook example for Compiler Design 101, but I’m afraid I’m lacking some formal education here. Is there a standard algorithm you can point me to?

My other options are to read up on Lexical Analysis or to roll up something quick and dirty with the available string functions.

Answer 1

As often, I would just use a regular expression to do this:

$expr = '(5*(7 + 2 * -9.3) - 8 )/ 11'; $tokens = preg_split('/([*\/^+-]+)\s*|([\d.]+)\s*/', $expr, -1,         PREG_SPLIT_DELIM_CAPTURE | PREG_SPLIT_NO_EMPTY); $tts = print_r($tokens, true); echo '<pre>x=$tts</pre>'; 

It needs a little more work to accept numbers with exponent (like -9.2e-8).