Question

Here are three pieces of similar code, the first two run fine, the last one fails —

    # 1
    print '%d) "%s" ' % (new_item_count, item[u'Title'].encode('utf-8')),
    print '%s' % item[u'CurrencyID'],
    print '%s !' % item[u'Value']

    # 2
    print '%d) "%s" %s%s !!' % (new_item_count,
            item[u'Title'].encode('utf-8'),
            item[u'CurrencyID'].encode('utf-8'),
            item[u'Value'])

    # 3
    print '%d) "%s" %s%s !!!' % (new_item_count,
            item[u'Title'].encode('utf-8'),
            item[u'CurrencyID'],
            item[u'Value'])

Output (note the differentiating exclamation marks at the end) —

37) "HP Pavilion Laptop / Intel® Core™ i3 Processor"  USD 510.0 !
37) "HP Pavilion Laptop / Intel® Core™ i3 Processor" USD510.0 !!
'ascii' codec can't decode byte 0xc2 in position 29: ordinal not in range(128)

This is under Python 2.6.5 (Ubuntu 10.04 package, same behaviour in both 32bit and 64bit) —

$ python -V
Python 2.6.5
$ uname -a
Linux <> 2.6.39.1-x86_64-<> #1 SMP Tue Jun 21 10:04:20 EDT 2011 x86_64 GNU/Linux

The only explanation I can think of is that it’s a Python bug.

Or is it?

Answer 1

You might even have triggered some incosisntency in there, but as it is, your code – in all of the examples. is somewhat messy.

You should not mix unicode and non-unicode strings like that

Please, tale a look at http://www.joelonsoftware.com/articles/Unicode.html – the article gives one a nice understanding of what unicode and different encodings actually are, and make it a lot easier to code anything afterwards.

As for your specific snippet, if all your data in the “item” dictionary is appropriately in unicode, as it seens, just do the string interpolation in unicode, and just encode the final resulting string to the intended output format:

message = u'%d) "%s" %s%s !!!' % (new_item_count,
            item[u'Title'],
            item[u'CurrencyID'],
            item[u'Value']))
print message.encode("utf-8")