Question

Possible Duplicate:
Converting a latin string to unicode in python

I have a list with the following format after storing in a file

list_example = [
         u"\u00cdndia, Tail\u00e2ndia & Cingapura",
         u"Lines through the days 1 (Arabic) \u0633\u0637\u0648\u0631 \u0639\u0628\u0631 \u0627\u0644\u0623\u064a\u0627\u0645 1",
]

But the actual format of the strings in the list is

actual_format = [
         "Índia, Tailândia & Cingapura ",
         "Lines through the days 1 (Arabic) سطور عبر الأيام 1 | شمس الدين خ "
]

How can I convert the strings in list_example to strings present in the actual_format list?

Answer 1

Your question is a bit unclear to me. In any case, the following guidelines should help you solving your problem.

If you define those strings in a Python source code, then you should

• know in which character encoding your editor saves the source code file (e.g. utf-8)
• declare that encoding in the first line of your source file, via e.g. # -*- coding: utf-8 -*-
• define those strings as unicode objects:

strings = [u"Índia, Tailândia & Cingapura ", u"Lines through the days 1 (Arabic) سطور عبر الأيام 1 | شمس الدين خ "]

(Note: in Python 3, literal strings are unicode objects by default, i.e. you don’t need the u. In Python 2, unicode strings are of type unicode, in Python 3, unicode strings are of type string.)

When you then want to save those strings to a file, you should explicitly define the character encoding:

with open('filename', 'w') as f:
    s = '\n'.join(strings)
    f.write(s.encode('utf-8'))

When you then want to read those strings again from that file, you again have to explicitly define the character encoding in order to decode the file contents properly:

with open('filename') as f:
    strings = [l.decode('utf-8') for line in f]