Question

It there a well known way, how to implement in Coco/R tokenizing indents like in Python/Boo?

Coco/R ignores whitespaces, but I need somehow to generate beginBlock/endBlock tokens, based on next line indent.

Right now, I use preprocessor, which inserts ‘{‘, ‘}’, and ‘;’ in input stream, by comparing indents between lines. In Coco/R grammar I use curved braces as beginBlock/endBlock tokens. It works well if input stream has no commens (which could be also nested). As soon as unordered comments coming, intentation comparison logic fails.

Implementing a preprocessor, which traces a comments looks like overenginering to me.

So the question is, is it generally possible to parse indent based grammar with Coco/R?
Or should I try something else?

Answer 1

Found a ideal way to do this.

• wrap GetNextToken with method that
  compares stream positions of the next
  token with the last one.

• if position.Y is changed, but
  position.X increased N tabs, inject N
  virtual INDENT tokens.

• if position.Y is changed, but
  position.X decreased N tabs, inject N
  virtual DENDENT tokens.

• if position.Y is changed, but
  position.X is not, inject virtual
  SEPARATOR token.

• if position.Y is not changed, return
  original next token.

• if previous token was a soft break (in python \),
  ignore logic above.