Question

I need to load any webpage using .net socket, so i can control how i connect and receive data myself.

Note

• Using WebClient and HttpWebRequest is not an option here, i need to
  use TcpClient and Socket.
• I’ll be happy if i could use them(HttpWebRequest) but control the connection and data myself.
• My real target is to just load any webpage using Socket, with my own HTTP rules.

Update

It’s possible to use WebClient and control the connection and data ourselves by running a C# HTTP proxy server and using the WebClient proxy address.
#Mentalis

Answer 1

Chunked transfer encoding is described pretty clearly in the RFC. Each chunk exists out of:

chunk-size[;chunk-extensions]<CRLF>
chunk-data<CRLF>

The chunk-size is sent first, which is a hexadecimal number which specifies how many bytes of chunk data to expect, optionally followed by chunk-extensions, proceeded by CRLF or \r\n. After you’ve read the specified amount of bytes, you can expect another CRLF, so you’ll have to read another two bytes.

Then you can start reading the next chunk. If chunk-size is 0, expect to read two more CRLF's (without adding it to your buffer) and then you have received all data since a chunk of size 0 indicates a last chunk.

Please note you cannot use ReadLine() to read chunks, since newlines in the response body (i.e. in chunk-data) will be seen as lines, so it’ll probably return before the whole chunk has been read.