Home/ Questions/Q 884869
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-15T12:48:53+00:00

I consume HTTP via nio.SocketChannel , so I get chunks of data as Array[Byte]

  • 0

I consume HTTP via nio.SocketChannel, so I get chunks of data as Array[Byte]. I want to put these chunks into a parser and continue parsing after each chunk has been put.

HTTP itself seems to use an ISO8859-Charset but the Payload/Body itself may be arbitrarily encoded:
If the HTTP Content-Length specifies X bytes, the UTF8-decoded Body may have much less Characters (1 Character may be represented in UTF8 by 2 bytes, etc).

So what is a good parsing strategy to honor an explicitly specified Content-Length and/or a Transfer-Encoding: Chunked which specifies a chunk-length to be honored.

  • append each data-chunk to an mutable.ArrayBuffer[Byte], search for CRLF in the bytes, decode everything from 0 until CRLF to String and match with Regular-Expressions like StatusRegex, HeaderRegex, etc?
  • decode each data-chunk with the proper charset (e.g. iso8859, utf8, etc) and add to StringBuilder. With this solution I am not able to honor any Content-Length or Chunk-Size, but.. do I have to care for it?
  • any other solution… ?
Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    I accumulate all Array[Byte] in an ArrayBuffer which allows me to count bytes.
    HTTP Protocol decoding (Status + Headers) is done by searching for the CRLF-position and then decoding 0 until CRLF with ISO8859.

    Chunked Bodies are accumulated in the ArrayBuffer and only decoded with the specified charset if the chunk has been fully saved in the ArrayBuffer. This circumvents MALFORMED exceptions from the CharsetDecoder if decoding utf8 data which is split right in the middle of a 2-byte character.

    For streaming HTML I have no good solution yet, normal HTML is buffered in the ArrayBuffer and decoded after the whole document has been received (like the chunks).

    • 0