Question

I am writing a C# interpreter from scratch for the learning experience, and so far everything has gone smoothly. I have a fully functional C# lexer that outputs all sorts of tokens to the parser. I know how I am going to parse the tokens, but I’m not sure how I should structure my AST (abstract syntax tree).

For example, if I have a simple code fragment:

using System.Xml;

what would the tree look like when parsed?

Like this?

UsingDirective
   Identifier(System)
      Identifier(Xml)

or like this?

UsingDirective
   Identifier(System)
   Identifier(Xml)

If I could get some suggestions and/or examples as to how I could structure things like identifiers with dots in them, if/else if/else statements, variable declaration/assignment combined in one statement (int i = 0;), function definitions, etc. that would be helpful. I just need to get a better idea of how to structure the tree and I can figure out the rest myself. Thanks.

Answer 1

I’ve written a couple of parsers in the past, and I would generally go for something like this:

UsingDirective
   IdentifierList
     Identifier (LeftNode) (System)
     Identifier (RightNode) (Xml)

In case of this using System.Collections.Generic

UsingDirective
   IdentifierList
       IdentifierList (LeftNode)
           Identifier (LeftNode) (System)
           Identifier (RightNode) (Collections)
       Identifier (RightNode) (Generic)

Unlike Roslyn, I prefer keeping my ASTs light by not including tokens such as semi-colon, the using keyword etc since the compiler has no need for them.

Parsers I’ve written specifically for IDEs look different – they carry all this extra stuff along with more information such as line and column numbers.