The following code uploads multiple files from a browser in a serial fashion to a host. In order to make the process faster, how could the files be processed in parallel so there are multiple streams being sent to the server? Is this worth pursuing if the files are being sent to the same host? Will a browser allow multiple upload streams to the same host? If so, how might this work?
I assume that is not possible to break the file into parts and write them parallel similar to this example since they are being submitted by a browser rather then a Python client.
#!/usr/bin/python
import cgi, os
import shutil
import cgitb; cgitb.enable() # for troubleshooting
form = cgi.FieldStorage()
print """\
Content-Type: text/html\n
<html><body>
"""
if 'file' in form:
filefield = form['file']
if not isinstance(filefield, list):
filefield = [filefield]
for fileitem in filefield:
if fileitem.filename:
fn = os.path.basename(fileitem.filename)
# save file
with open('/var/www/site/files/' + fn, 'wb') as f:
shutil.copyfileobj(fileitem.file, f)
# line breaks are not occuring between interations
print 'File "' + fn + '" was uploaded successfully<br/>'
message = 'All files uploaded'
else:
message = 'No file was uploaded'
print """
<p>%s</p>
</body></html>
""" % (message)
A form submit produces a single HTTP request that naturally uses a single TCP connection to deliver the request.
To upload multiple files in parallel make multiple requests in parallel. You might need flash or java applet on the client. Check whether javascript (AJAX) allows concurrent multiple form submissions.