Question

I’m having some trouble making my website compatible with accented characters (french website).

I have a form where some field values can be with accented chars: “Coupé” for instance.

My URL looks like this:

http://localhost:8080/recherches/s?marque=Audi&modeles=A5+Coup%C3%A9

In my django view I do something like this:

def search(request):
  logger = logging.getLogger('custom')
  criteria_form = CriteriaForm(request.GET or None)
  logger.debug("search")
  logger.debug(request.GET)

And what I get in my logs is:

<QueryDict: {u'marque': [u'Audi'], u'modeles': [u'A5 Coup\xc3\xa9']}>

If I query my database with this variable “modeles”, I get an error:

>>> mo = u'A5 Coup\xc3\xa9'
>>> Vehicule.objects.filter(valid=True, modele=mo)[0].marque.name
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/django/db/models/query.py", line 211, in __getitem__
    return list(qs)[0]
IndexError: list index out of range

Things work if I query the database with the utf-8 version:

>>> mo = 'A5 Coup\xc3\xa9'
>>> Vehicule.objects.filter(valid=True, modele=mo)[0].marque.name
u'Audi'

So I think (but I might be wrong) that my problem comes from the fact that my variable is utf8 and then encoded with unicode.

How comes this is encoded that way?

UPDATE AFTER 1st RESPONSE:

On the header of the page that sends the form there is:

<meta http-equiv="Content-type" content="text/html;charset=UTF-8"/>

However if I print in my views.py the encoding:

logger.debug(request.encoding)

Then I get None.

But I don’t know how to setup this encoding. I thought it would be from the header like I did above…

Also I have that in my HTTP_ACCEPT_CHARSET:

HTTP_ACCEPT_CHARSET ISO-8859-1,utf-8;q=0.7,*;q=0.3

Can that come from here? If yes, how should I change that?

Answer 1

I have finally found the problem. I tested this problem with a brand new django application with nearly nothing in it. No DB, a simple view which displayed the content of the form in a page.
Something like this:

# -*- coding: utf-8 -*-
from django.shortcuts import render_to_response
from django.template import RequestContext


def test(request):
  found = request.GET.get('modeles')
  print found
  return render_to_response('test.html',
                            {"found":found},
                            context_instance=RequestContext(request))

And if I opened the url

localhost:8080/mysite?modeles=Coupé

I was getting the wrongly formatted Coupé

Here for sure I could not blame any ajax, or db call or anything I could have done in python.

So I just tried it with Django 1.4… And it worked like a charm!
Then I tried with the latest Django 1.5a1 version and it also worked…

I guess I should not use the beta for now. I will stick with the 1.4!

Hope this saves time for somebody else.