Question

I’d like to create my own custom HTTP requests. The WebClient class is very cool, but it creates the HTTP requests automatically. I’m thinking I need to create a network connection to the web server and pass my data over that stream, but I’m unfamiliar with the library classes that would support that kind of thing.

(Context, I’m working on some code for a web programming class that I’m teaching. I want my students to understand the basics of what’s happening inside the “black box” of HTTP.)

Answer 1

To really understand the internals of the HTTP protocol you could use TcpClient class:

using (var client = new TcpClient("www.google.com", 80))
{
    using (var stream = client.GetStream())
    using (var writer = new StreamWriter(stream))
    using (var reader = new StreamReader(stream))
    {
        writer.AutoFlush = true;
        // Send request headers
        writer.WriteLine("GET / HTTP/1.1");
        writer.WriteLine("Host: www.google.com:80");
        writer.WriteLine("Connection: close");
        writer.WriteLine();
        writer.WriteLine();

        // Read the response from server
        Console.WriteLine(reader.ReadToEnd());
    }
}

Another possibility is to activate tracing by putting the following into your app.config and just use WebClient to perform an HTTP request:

<configuration>
  <system.diagnostics>
    <sources>
      <source name="System.Net" tracemode="protocolonly">
        <listeners>
          <add name="System.Net"/>
        </listeners>
      </source>
    </sources>
    <switches>
      <add name="System.Net" value="Verbose"/>
    </switches>
    <sharedListeners>
      <add name="System.Net"
           type="System.Diagnostics.TextWriterTraceListener"
           initializeData="network.log" />
    </sharedListeners>
    <trace autoflush="true"/>
  </system.diagnostics>
</configuration>

Then you can perform an HTTP call:

using (var client = new WebClient())
{
    var result = client.DownloadString("http://www.google.com");
}

And finally analyze the network traffic in the generated network.log file. WebClient will also follow HTTP redirects.