Question

# -*- coding: utf-8 -*-
from pyquery import PyQuery as pq
from urllib import urlencode
from urllib2 import Request,urlopen
    def sendRequest(url, data = None, headersOnly = False):
    headers  = { 'User-Agent' : 'Mozilla/5.0 (X11; U; Linux i686; en-US;)' }
    request  = Request(url, data, headers)
    return urlopen(request).read()

resp = sendRequest("https://foursquare.com/v/rivers-edge-cafe--    morrison/4c1907776e02b7132eae627b")
print pq(resp)("#venueCategories").text()

Output should be Café, Burger Joint, Sandwich Place
but got exception:

Traceback (most recent call last):
  File "unicodeerr1.py", line 11, in <module>
    print pq(resp)("#venueCategories").text()
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 3: ordinal not in range(128)

Answer 1

You need to specify an encoding for your source code, or use a character escape instead.

# -*- coding: utf-8 -*-

or

print 'caf\xc3\xa9' # UTF-8 representation of e accent egu.

You probably want to use a Unicode literal though (here with a unicode escape character):

print u'caf\u00e9'

Please do read the Unicode HOWTO to fully understand what goes on here. Other helpful documents:

• Joel Spolsky’s The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)
• Ned Batchelder’s Pragmatic Unicode presentation.

Note that your specific error doesn’t have anything to do with python 2.5 vs. 2.7, but everything with the output encoding of whatever you are printing to. On your server with python2.5, there is either no encoding specified or it is explicitly set to ASCII, but on your local machine with python 2.7, you are most likely dealing with a terminal that supports UTF-8.