Question

I am writing an http automation framework and my problem is communicating with an authenticated http secured connection. After doing some research, I discovered the SslStream object in C# which made for easy integration with my existing client architecture. The problem is; despite being able to authenticate my connection with a given web server, any “GET [website] HTTP/1.1” commands seem to only return headers and not actual web pages.

I have a feeling that I am not forming my web requests properly, but I really don’t know. I have been doing research all morning and cannot find very many resources on this particular issue. Here is a code sample from my project:

    private IAsyncResult Request(HttpRequestToken token, ReceiveEvent callback) 
    {
        if (token == null)
            throw new Exception("Error. No request information provided. Aborting operation.");

        //Setup the TCP Information. (_port is set to 443 for SSL purposes)
        var client = new TcpClient(token.Host, _port);

        //Get a handle to a networkstream for writing data.
        var requestStream = new SslStream(client.GetStream(), false, null);

        //Authenticate the request
        requestStream.AuthenticateAsClient(token.Host);

        //Translate the data.
        byte[] sendBuffer = UTF8Encoding.UTF8.GetBytes(token.ToString());

        //NOTE: The results of the above command will look like this:
        //GET [website] HTTP/1.1
        //Host: [host]
        //passive: true
        //Accepts: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
        //User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1 Fennec/2.0.1

        //Send the data.
        requestStream.Write(sendBuffer);

        //Get the string value with a function that just iterates over the stream.
        string val = readStream(requestStream);

        //Breakpoint here, this code is not complete but "functions".
        return null;
    }

Essentially, the results of my code will just return an “HTTP/1.1 200 OK” with some header information. No HTML or anything else is returned beyond the headers.

As Requested, Here is some more info:

    private string readStream(Stream stream)
    {
        byte[] resultBuffer = new byte[2048];
        string value = "";
        //requestStream.BeginRead(resultBuffer, 0, resultBuffer.Length, new AsyncCallback(ReadAsyncCallback), new result() { buffer = resultBuffer, stream = requestStream, handler = callback, asyncResult = null });
        do
        {
            try
            {
                int read = stream.Read(resultBuffer, 0, resultBuffer.Length);
                value += UTF8Encoding.UTF8.GetString(resultBuffer, 0, read);

                if (read < resultBuffer.Length)
                    break;
            }
            catch { break; }
        } while (true);
        return value;
    }

For testing purposes, I am trying to access the google android developer portal (since it uses SSL). This is simply to load the login page, no information is even being transmitted at this point other than a page load request. Here is my exact request.

GET https://accounts.google.com/ServiceLogin HTTP/1.1
Host: accounts.google.com
passive: true
nui: 1
continue: https://market.android.com/publish
followup: https://market.android.com/publish
Accepts: text/html,application/xhtml+xml,application/xml;q=0.9,
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1 Fennec/2.0.1

This is the server response:

HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Strict-Transport-Security: max-age=2592000; includeSubDomains
Set-Cookie: GAPS=1:0bFTJDze2Zz8WL_x3F7-OQfOjEOycg:rG8nLpBEwdG67aU_;Path=/;Expires=Mon, 27-Jan-2014 21:31:48 GMT;Secure;HttpOnly
Set-Cookie: GALX=KaXGmr2TI-I;Path=/;Secure
Cache-control: no-cache, no-store
Pragma: no-cache
Expires: Mon, 01-Jan-1990 00:00:00 GMT
X-Frame-Options: Deny
X-Auto-Login: realm=com.google&args=continue%3Dhttps%253A%252F%252Faccounts.google.com%252FManageAccount
Transfer-Encoding: chunked
Date: Sat, 28 Jan 2012 21:31:48 GMT
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Server: GSE

Thank you for taking the time to look at my question, I appreciate it! If you would like more information from me, I am happy to give it. Since I believe my mistake is probably a silly format-issue I didn’t think more information would be required.

Again, thanks!

Answer 1

The readStream function terminates when it reads less than a full buffer.

However, this just means that there is no more data available now; there might be more available in the future.

Don’t stop reading until stream.Read returns zero bytes.