Home/ Questions/Q 8386645
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-09T17:59:56+00:00

I am learning the HTML parsing. During the tokenization stage, the byte stream is

  • 0

I am learning the HTML parsing. During the tokenization stage, the byte stream is tokenized into tokens. How many token types does the standard HTML tokenization support? Does it include something like start tag token, comment token ?

Will the comments be considered as tokens and attached to the DOM tree?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    The HTML specification says:

    The output of the tokenization step is a series of zero or more of the following tokens: DOCTYPE, start tag, end tag, comment, character, end-of-file.

    So there are six different tokens.

    To answer your last question: comments are tokens and there is a DOM interface for them.

    • 0