Question

Internally ASP.NET has a 2 GB addressing space, but in reality you only have less than 1 GB free for uploads (see http://support.microsoft.com/?id=295626 ). In addition IIS 7 has a cap of 30 MB ( see http://www.iislogs.com/steveschofield/iis7-post-40-adjusting-file-upload-size-in-iis7 ) and you supposedly have to run

appcmd set config "My Site/MyApp" -section:requestFiltering -requestLimits.maxAllowedContentLength:104857600 -commitpath:apphost

on the server to go beyond this 30 MB limit. But how can I run this on my Azure servers?

Also, according to http://support.microsoft.com/?id=295626

During the upload process, ASP.NET
loads the whole file in memory before
the user can save the file to the
disk.

, so I will quickly exhaust the memory limit if many users upload large files at one time. In my code below I use streams, but I’m guessing that the whole file is uploaded in memory first anyway. Is this the case?

using System;
using System.Web.Security;
using Microsoft.WindowsAzure;
using Microsoft.WindowsAzure.StorageClient;

namespace WebPages
{
    public partial class Upload : System.Web.UI.Page
    {
        CloudBlobClient BlobClient = null;
        CloudBlobContainer BlobContainer = null;

        void InitBlob()
        {
            // Setup the connection to Windows Azure Storage
            var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
            BlobClient = storageAccount.CreateCloudBlobClient();

            // Get and create the container
            BlobContainer = BlobClient.GetContainerReference("publicfiles");
        }

        protected void Page_Load(object sender, EventArgs e)
        {
            //if (Membership.GetUser() == null) return;   // Only allow registered users to upload files

            InitBlob();

            try
            {
                var file = Request.Files["Filedata"];

                var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
                BlobClient = storageAccount.CreateCloudBlobClient();

                // Make a unique blob name
                var extension = System.IO.Path.GetExtension(file.FileName);

                // Create the Blob and upload the file
                var blobAddressUri = String.Format("{0}{1}", Guid.NewGuid(), extension);
                var blob = BlobContainer.GetBlobReference(blobAddressUri);

                blob.UploadFromStream(file.InputStream);

                // Set the metadata into the blob
                blob.Metadata["FileName"] = file.FileName;
                //blob.Metadata["Submitter"] = Membership.GetUser().UserName;
                blob.Metadata["Type"] = "Video";
                blob.Metadata["Description"] = "Test";
                blob.SetMetadata();

                // Set the properties
                blob.Properties.ContentType = file.ContentType;
                blob.SetProperties();
            }
            catch(Exception ex)
            {
                System.Diagnostics.Trace.TraceError("Upload file exception: {0}", ex.ToString());
                // If any kind of error occurs return a 500 Internal Server error
                Response.StatusCode = 500;
                Response.Write("An error occured while uploading the file");
                Response.End();
            }
        }
    }
}

I am aware of non web page upload tools like http://azureblobuploader.codeplex.com/ , but I really need it to be uploaded from a web page.

So, my questions are:

1. How to I upload files to the blob that are larger than 2 GB from a web page
2. How do I large upload files from a web page as a stream that does not eat all the memory
3. If the solution is to write my own HttpModule or HttpHandler to handle my upload, how can I get this installed on my Azure servers? Can I use HttpHandlers like http://neatupload.codeplex.com/ on Azure?
4. This project is not on SharePoint, but I knwo that in SharePoint you have something called a Blob Provider and that you can write your own, are there Blob Providers for ASP.NET?

I can also mention that my code above works fine by default with files smaller than 30 MB, I use SWFUpload V2.2.0 on the client.

Update 19. June 19:09:
@YvesGoeleven on Twitter gave me a tip of using Shared Access Signature (see msdn.microsoft.com/en-us/library/ee395415.aspx ) and uploading the file directly to the Azure Blob Storage without going through the ASP.NET at all. I created a JSON WCF that returns a valid SAS ut to my blob storage.

using System.ServiceModel;
using System.ServiceModel.Web;

namespace WebPages.Interfaces
{
    [ServiceContract]
    public interface IUpload
    {
        [OperationContract]
        [WebInvoke(Method = "GET",
            ResponseFormat = WebMessageFormat.Json)]
        string GetUploadUrl();
    }
}

--------

using System;
using System.IO;
using System.Runtime.Serialization.Json;
using System.ServiceModel.Activation;
using System.Text;
using Microsoft.WindowsAzure;
using Microsoft.WindowsAzure.StorageClient;

namespace WebPages.Interfaces
{
    [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
    public class UploadService : IUpload
    {
        CloudBlobClient BlobClient;
        CloudBlobContainer BlobContainer;

        public UploadService()
        {
            // Setup the connection to Windows Azure Storage
            var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
            BlobClient = storageAccount.CreateCloudBlobClient();

            // Get and create the container
            BlobContainer = BlobClient.GetContainerReference("publicfiles");
        }

        string JsonSerialize(string url)
        {
            var serializer = new DataContractJsonSerializer(url.GetType());
            var memoryStream = new MemoryStream();

            serializer.WriteObject(memoryStream, url);

            return Encoding.Default.GetString(memoryStream.ToArray());
        }

        public string GetUploadUrl()
        {
            var sasWithIdentifier = BlobContainer.GetSharedAccessSignature(new SharedAccessPolicy()
            {
                Permissions = SharedAccessPermissions.Write,
                SharedAccessExpiryTime =
                    DateTime.UtcNow.AddMinutes(60)
            });
            return JsonSerialize(BlobContainer.Uri.AbsoluteUri + "/" + Guid.NewGuid() + sasWithIdentifier);
        }
    }
}

It works, but I can’t use it with SWFUpload since it uses the HTTP POST verb and not the HTTP PUT verb that the Azure Blob Storage expects when creating a new blob item. Anyone know how to get around this without making my own custom Silverlight or Flash client component that uses the HTTP PUT verb? I wanted a progress bar when uploading the files, therefore a submitted form that uses PUT is not optimal.

For those interested in the client code (that wont work since SWFUpload uses HTTP POST and not PUT as Azure Blob Storage expects):

    <div id="header">
        <h1 id="logo"><a href="/">SWFUpload</a></h1>
        <div id="version">v2.2.0</div>
    </div>
    <div id="content">
        <h2>Application Demo (ASP.Net 2.0)</h2>
        <div id="swfu_container" style="margin: 0px 10px;">
            <div>
                <span id="spanButtonPlaceholder"></span>
            </div>
            <div id="divFileProgressContainer" style="height: 75px;"></div>
            <div id="thumbnails"></div>
        </div>
    </div>

 <script type="text/javascript" language="javascript">
        $(document).ready(function () {

            $.ajax({
                url: '/Interfaces/UploadService.svc/GetUploadUrl',
                success: function (result) {
                    var parsedResult = $.parseJSON(result);
                    InitUploadFile(parsedResult);
                }
            });


            function InitUploadFile(uploadUrl) {
                //alert(uploadUrl);
                var swfu = new SWFUpload({
                    // Backend Settings
                    upload_url: uploadUrl,
                    post_params: {
                        "ASPSESSID": "<%=Session.SessionID %>"
                    },

                    // File Upload Settings
                    file_size_limit: "100 MB",
                    file_types: "*.*",
                    file_types_description: "All file types",
                    file_upload_limit: "0",    // Zero means unlimited

                    // Event Handler Settings - these functions as defined in Handlers.js
                    //  The handlers are not part of SWFUpload but are part of my website and control how
                    //  my website reacts to the SWFUpload events.
                    file_queue_error_handler: fileQueueError,
                    file_dialog_complete_handler: fileDialogComplete,
                    upload_progress_handler: uploadProgress,
                    upload_error_handler: uploadError,
                    upload_success_handler: uploadSuccess,
                    upload_complete_handler: uploadComplete,

                    // Button settings
                    button_image_url: "Images/swfupload/XPButtonNoText_160x22.png",
                    button_placeholder_id: "spanButtonPlaceholder",
                    button_width: 160,
                    button_height: 22,
                    button_text: '<span class="button">Select files <span class="buttonSmall">(2 MB Max)</span></span>',
                    button_text_style: '.button { font-family: Helvetica, Arial, sans-serif; font-size: 14pt; } .buttonSmall { font-size: 10pt; }',
                    button_text_top_padding: 1,
                    button_text_left_padding: 5,

                    // Flash Settings
                    flash_url: "Js/swfupload-2.2.0/swfupload.swf", // Relative to this file

                    custom_settings: {
                        upload_target: "divFileProgressContainer"
                    },

                    // Debug Settings
                    debug: false
                });
            }
       });
    </script>

Update 19. June 21:07:

I figured since SWFUpload is open source that I download the source and change the verb from POST to PUT, sadly the Flash Player URLRequestMethod does not support other verbs than GET and POST. I did find a supposed work-around

private function BuildRequest():URLRequest {
   // Create the request object
   var request:URLRequest = new URLRequest();
   request.method = URLRequestMethod.POST;
   request.requestHeaders.push(new URLRequestHeader("X-HTTP-Method-Override", "PUT"));

, but this only work in Adobe Air and not with the Flash Player.

I’ve read that SilverLight 3 and later supports the HTTP PUT verb, so I think I have to write some SilverLight code to get my way here. I did find this blog article series that will probably help me here http://blog.smarx.com/posts/uploading-windows-azure-blobs-from-silverlight-part-1-shared-access-signatures .

Update @ 27. June ’11:

I now have successfully managed to upload large files (tested with 4,5 Gb files) from a web page using a custom Silverlight client I wrote based on the project in http://blog.smarx.com/posts/uploading-windows-azure-blobs-from-silverlight-part-1-shared-access-signatures . Since Silverlight supports both the HTTP PUT verb that Azure Blob Storage requires and supports progressive uploads, I now have the possibility to upload massive files directly to the Azure Blob Storage and I don’t have to go throgh a ASP.NET solution, I also get some nice progress bars and the user can cancel in the middle of the upload if he/she wants to. The memory usage on the server is minimal since the whole file is not uploaded before it is placed in the Azure Blob Storage. I use a Shared Access Signature (see msdn.microsoft.com/en-us/library/ee395415.aspx ) that is supplied from a WCF RESTfull service on request. I think that this solution is the best one we found. Thanks.

Update @ 18. July ’11:

I have created an open source project with what I found here:

http://azureslfileuploader.codeplex.com/

Answer 1

I actually did the exact same thing recently. I created a Silverlight Client app to handle chopping up the data and sending it to Azure.

This is a working example that I followed that does exactly that. Pretty much follow this and you’re work is almost much done for you.