Question

I am sending encoded post data to appengine project.But appengine shows blank char in response.

I am sending this form data :
mdata=I%FD%FD%FE%DE%DE%E7%E7%C7%D6%F6%F6%D6%D6%0A%0A

and my appengine response code is :

request_data = self.request.get('mdata')
mailhtmldata = urllib.unquote_plus(request_data)

And repr(request_data) and repr(mailhtmldata) is u'I\n\n' .Still unicode chars like “İşÇöÖü” are blank.I am sure about my sender is sending mdata=I%FD%FD%FE%DE%DE%E7%E7%C7%D6%F6%F6%D6%D6%0A%0A.i tested.Where is other chars in appengine ?

Thanks for all help.

Sorry for my bad english

Update :

This is my sender function

def mailSend():
    values = urllib.urlencode({'mailam' : '[e-mail]', 'mfromname' :'Deneme Kisisi', 'mkonu' : 'This is Subject', 'mdata' : 'IıışŞşÖÖççç'})
    headers = {"Content-type": "application/x-www-form-urlencoded", "Accept": "text/plain"}
    conn = httplib.HTTPConnection("xxxxxx.appspot.com")
    conn.request("POST", "/gondergitsin", values, headers)
    response = conn.getresponse()
    data = response.read()
    print data
    conn.close()

And this is the appengine code:

class mTransfer(webapp.RequestHandler):
    def post(self):
        mailhtmldata = urllib.unquote_plus(self.request.get('mdata'))
        x2adresx = urllib.unquote(self.request.get('mailam'))
        x2gonderenx = urllib.unquote(self.request.get('mfromname'))
        x2gondid = ''.join(random.choice(string.ascii_lowercase + string.digits) for x in range(12))
        anagonderen = "Deneme <[e-mail]>"
        mailsubjdata = urllib.unquote(self.request.get('mkonu'))
        message = mail.EmailMessage(sender=anagonderen, subject=mailsubjdata)
        message.to = x2adresx
        message.body = mailhtmldata
        message.html = mailhtmldata
        message.send()
        self.response.out.write("OK")

I tried but still i can’t found a solution

Update 2: If you are using unicode charsets you decode it to your charset and after that you can encode it to utf-8.

Answer 1

What you are sending appears to be the result of url-encoding some characters expressed in cp1254 (Windows Turkish) encoding (or the very similar ISO-8859-9 encoding). Have you told “appengine” what encoding you are using?

We need a bit more information about “Still unicode chars like “İşÇöÖü” are blank”. Blank when you look at them how? Instead of

htmldata = urllib.unquote_plus(self.request.get('mdata'))

do this:

request_data = self.request.get('mdata')
htmldata = urllib.unquote_plus(request_data)

and edit your question to show the results of repr(request_data) and repr(htmldata)

Update You say that self.request.get('mdata') is returning u'I\n\n' … the non-ASCII characters in your data are not coming back as blank; they are being removed.

Somebody’s code is stripping out non-ASCII characters. Something like your_input.decode(some_encoding, 'ignore') with some_encoding set to ascii or UTF-8.

Show us the code where you “told to appengine what encoding i am using(iso-8859-9)”.

Update 2 in response to posting of sending code and this comment “I am adding # -*- coding: iso-8859-9 -*- to top”.

Putting the “coding” thing at the top of your source file is telling the Python compiler the encoding of the remainder of your source file. It has nothing to do with the encoding of your data. You could remove the # coding thing and instead of writing 'mdata' : 'IıışŞşÖÖççç' you could write 'mdata' : 'I\xFD\xFD etc etc' and get the same string of bytes sent to the server with the same effect. You have NOT told appengine what encoding you are using.

Instead of

"Content-type": "application/x-www-form-urlencoded",

I suggest that you try this:

"Content-Type": "application/x-www-form-urlencoded; charset:ISO-8859-9;"