Question

Within my project, when I am submitting some data with non-ascii characters to the form, I am getting all non-ascii data replaced with unicode replacement character \ufffd. In the view that receives the data from the form, I have this malformed strings in request.POST. Seems I’ve missed something obvious.

My environment:

• Python 2.6
• Django 1.3 alpha 1
• MySQL 5.1, database uses UTF-8 charset

Have reproduced this issue using development server locally and intermediate server running Apache+mod_wsgi, with Firefox 4 and Chrome 11.

Where should I look to workout the problem? Thanks.

Update: below is the code I am using –

@render_to('mail/new_message.html')
@login_required
def new_message(request, user_id):
    user = request.user

    if request.method == 'POST':
        form = MessageForm(request.POST, request.FILES)
        # exclude attachment from init instance, because we need instance id
        # for saving attachment
        form._meta.exclude = ['attachment']
        if form.is_valid():
            new_msg = form.save()
            new_msg.sender = user
            if form.cleaned_data['attachment']:
                new_msg.attachment = form.cleaned_data['attachment']
            new_msg.save()
            message_sent.send(sender=None,instance=new_msg)
            return HttpResponseRedirect(reverse(messages))
    else:
        initial = {}
        if user_id:
            initial['receivers'] = [user_id,]
        form = MessageForm(initial=initial)
    return {'form': form}

Answer 1

Solved: I forgot to add encoding to HTML pages I am generating. Adding of <meta http-equiv="content-type" content="text/html; charset=utf-8"> solved the problem.