Question

I am executing a web request using the following:

            // create http request
            HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
            // set user agent
            req.UserAgent = "Mozilla/5.0 (X11; U; Linux i586; de; rv:5.0) Gecko/20100101 Firefox/5.0";
            req.AllowAutoRedirect = true;
            req.MaximumAutomaticRedirections = 3;
            // retrieve response
            HttpWebResponse rep = (HttpWebResponse)req.GetResponse();
            HttpStatusCode  stc = rep.StatusCode;

On some URLs, but not all, the status returns OK with -1 content length, but if I use Firefox directly it renders.

Is there a setting that I need to provide to C#? Sample link http://www.cbsstore.com/detail.php?p=382214&ecid=5511&pa=CSE-FGL&CAWELAID=1599172025

Answer 1

If you’re getting the content length from HttpWebResponse.ContentLength, a value of -1 doesn’t mean there’s no content – it means “keep reading until you get to the end of the stream” (because the Content-Length header hasn’t been set). A content length of zero would mean no content.

Basically, if you want to see the content, you should actually try to read it 🙂