Question

I need to upload and read a CSV then save to database. I’m a beginner and below is what I have achieved so far using ‘django-adaptors’ (http://django-adaptors.readthedocs.org/en/latest/index.html) I know its not a lot but I’m just doing this to learn more 🙂

I’m struggling in the view bit of my code (below). I don’t know how to upload then read the file into to CodeCSvModel() function? Could anyone help explain? Many thanks. 🙂

views.py

from django.template import RequestContext
from django.shortcuts import render_to_response
from web.forms import codeUploadForm
from web.csvTools import CodeCSvModel


def codeImport(request):
    # If we had a POST then get the request post values.
    if request.method == 'POST':
        form = codeUploadForm(request.POST, request.FILES)
      # handle_uploaded_file(request.FILES['file'])

                ====[HELP HERE]=====

        #form = codeUploadForm(request.POST)

        CodeCSvModel.import_from_file(form['file'])



    else:
         form = codeUploadForm()
         context = {'form':form}
         return render_to_response('import.html', context, context_instance=RequestContext(request))  

forms.py

class codeUploadForm(forms.Form):

    file = forms.FileField()
    place = forms.ModelChoiceField(queryset=Incentive.objects.all())

csvTool.py

from datetime import datetime
from adaptor.fields import *
from adaptor.model import CsvModel, CsvDbModel, ImproperlyConfigured,\
    CsvException, CsvDataException, TabularLayout, SkipRow,\
    GroupedCsvModel, CsvFieldDataException
from web.models import *



class CodeCSvModel(CsvModel):

    codeid = CharField()
    remotecode = CharField()
    active = BooleanField()
    created = DateField()
    modified = DateField()
    incentiveid = CharField()

    class Meta:
        delimiter = ";"
        dbModel = Code

Answer 1

Django only keeps the uploaded file in memory if it is 2.5MB or smaller, otherwise it writes to a temporary location.

The step you’re missing is where you actually write the file from memory/temporary location to disk somewhere, e.g.

if request.method == 'POST':
    form = MyForm(request.POST, request.FILES)
    if form.is_valid():
        uploaded_file = request.FILES['html-file-attribute-name']

        # Write the file to disk
        fout = open("path/to/save/file/to/%s" % uploaded_file.name, 'wb')
        for chunk in uploaded_file.chunks():
            fout.write(chunk)
        fout.close()

Chunking is the way to go if the file is very large, because Django will read each chunk of the file into memory before writing to disk. If you use read() instead, it will read the entire file into memory, so chunks is your best bet.