Question

I am tasked with writing a report that pulls data out of a the “Storage Space Allocation” section within various SharePoint sites. I am able to screen scrape the general “Document Libraries” values by performing a general GET call, but I cannot programmatically obtain the “Lists” values. When I navigate to the SharePoint site (*/_layouts/storman.aspx) “Document Libraries” is the default selection. I think I need to send a POST call in order to change it to “Lists” [then I can scrape the values]. Creating the appropriate POST call is becomine a hassle because SharePoint does not seem to recognize my key/value pair (or maybe I’m not supplying all of the necessary parameters?).

I tried this code, but no luck – only the “Document Libraries” data is returned.

using (System.Net.WebClient client = new System.Net.WebClient() { UseDefaultCredentials = true })
{
    NameValueCollection myQueryStringCollection = new NameValueCollection();
    myQueryStringCollection.Add(queryParameterName, queryParameterValue);

    client.QueryString = myQueryStringCollection;

    return client.DownloadString(url);
}

I also tried this (alongside other ideas):

private static string GetWebResponse(string url, NameValueCollection parameters)
{
    var httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
    httpWebRequest.UseDefaultCredentials = true;
    httpWebRequest.ContentType = "application/x-www-form-urlencoded";
    httpWebRequest.Method = "POST";

    var sb = new StringBuilder();
    foreach (var key in parameters.AllKeys)
        sb.Append(key + "=" + parameters[key] + "&");
    sb.Length = sb.Length - 1;

    byte[] requestBytes = Encoding.UTF8.GetBytes(sb.ToString());
    httpWebRequest.ContentLength = requestBytes.Length;

    using (var requestStream = httpWebRequest.GetRequestStream())
    {
        requestStream.Write(requestBytes, 0, requestBytes.Length);
        requestStream.Close();
    }

    Task<WebResponse> responseTask = Task.Factory.FromAsync<WebResponse>(httpWebRequest.BeginGetResponse, httpWebRequest.EndGetResponse, null);
    using (var responseStream = responseTask.Result.GetResponseStream())
    {
        var reader = new StreamReader(responseStream);
        return reader.ReadToEnd();
    }
}

Viewing the source code of the _layouts/storman.aspx page, I can see the name/value pair i need to send is ct100$PlaceHolderMain$m_filterDropdown and Lists respectively. I determined this by this view source code:

<select name="ctl00$PlaceHolderMain$m_filterDropdown" id="ctl00_PlaceHolderMain_m_filterDropdown" class="ms-viewselect">
<option selected="selected" value="Document Libraries">Document Libraries</option>
<option value="Documents">Documents</option>
<option value="Lists">Lists</option>
<option value="Recycle Bin">Recycle Bin</option>
</select>

Any ideas on how to get the List values from this page?

Answer 1

I finally figured this out…

I have to send every name/value query parameter pair that SharePoint is expecting. To complicate matters, I must first perform a GET request in order to obtain the __REQUESTDIGEST, __VIEWSTATE, __EVENTVALIDATION values that SharePoint is expecting. Also, all parameters must be UrlEncoded when I send them back to SharePoint.