Question

I’ve made a servlet which uses the org.apache.commons.fileupload api to upload a CSV file which it should then load into a MySQL table. This works perfectly when posting to the servlet from a form in a browser. However, it fails when trying to post a form via a Tcl script http://www.tcl.tk/man/tcl/TclCmd/http.htm#M20.

The servlet throws the following MalformedStreamException:

org.apache.commons.fileupload.MultipartStream$MalformedStreamException:
Stream ended unexpectedly

The servlet is hosted on Tomcat version 6.0.16.

The connection is made successfully by the Tcl script as it receives an HTTP/1.1 200 OK response and the servlet does return some print statements back to the client Tcl script. However it fails when trying to read the input stream.

Tcl script:

package require http

proc upload { url csv } {
    set boundary "-----WebKitFormBoundary[clock seconds][pid]"

    set fid [open $csv r]
    if {[catch {read $fid [file size $csv]} data]} {
        return -code error $data
    }
    close $fid

    set content {}
    append content "--${boundary}\n"
    append content "Content-Disposition: form-data; name=\"db\"\n\n"
    append content "test\n"
    append content "--${boundary}\n"
    append content "Content-Disposition: form-data; name=\"table\"\n\n"
    append content "testing\n"
    append content "--${boundary}\n"
    append content "Content-Disposition: form-data; name=\"file\"; filename=\"$csv\"\n"
    append content "Content-Type: text/csv\n\n"
    append content "$data\n"
    append content "${boundary}--"

    set headers "connection keep-alive"
    set token [::http::geturl $url -keepalive 1 -headers $headers -query $content -type "multipart/form-data; boundary=$boundary"]

    upvar 0 $token state

    if {$state(http) == "HTTP/1.1 200 OK"} {
        # no error reported in http headers
        puts stdout $state(http)
        puts stdout $state(body)
        return 1
    } else {
        # error reported in http headers
        puts stdout $state(http)
        puts stdout $state(body)
        return 0
    }
}

set csv "data.csv"
set url "http://ecat:8080/MySqlImport/MySqlImportServlet"
set retVal [upload $url $csv]

Answer 1

Thank you for the help on this one but I have found my specific issue and it was a simple one in the end. It seems as though the ServletFileUpload.parseRequest(request) method in apache.commons.fileupload requires the line endings to be in windows format.

As you can see in the question the $content line endings was \n changing this to \r\n corrected the problem.

Using just \r meant that the error

org.apache.commons.fileupload.MultipartStream$MalformedStreamException:
Stream ended unexpectedly

no longer occured, however it failed to recognise the different items or parts in the request. The combination of \r\n line endings and it works correctly.