Question

I am trying to return JSON from the API service from musicbrainz, the returned data for some songs have unicode characters which I am having trouble converting them to regular symbols etc. Kindly let me know what I should be doing here.

JSON:

{
    "status": "ok",
    "results": [{
        "recordings": [{
            "duration": 402,
            "tracks": [{
                "duration": 402,
                "position": 6,
                "medium": {
                    "release": {
                        "id": "dde6ecee-8e9b-4b46-8c28-0f8d659f83ac",
                        "title": "Tecno Fes, Volume 2"
                    },
                    "position": 1,
                    "track_count": 11
                },
                "artists": [{
                    "id": "57c1e5ea-e08f-413a-bcb1-f4e4b675bead",
                    "name": "Gigi D\u2019Agostino"
                }],
                "title": "You Spin Me Round"
            }],
            "id": "2e0a7bce-9e44-4a63-a789-e8c4d2a12af9"
        }, ....

Failed Code (example):

string = '\u0420\u043e\u0441\u0441\u0438\u044f'
print string.encode('utf-8')

I am using this on a windows 7 machine and have python 2.7 and running this code on a command line terminal.. I have the output I get below:

C:\Python27>python junk.py Gigi DGÇÖAgostino Gigi D?Agostino Gigi D\u2019Agostino

I am expecting the output to be Gigi D' Agostino

Answer 1

You are using the cmd in Windows? In that case it might be a bit of a hack to get Unicode working at all to display correctly.
You might want to think about using another “terminal” to test your scripts.
MSYS provides a nice terminal/shell and IDLE is included in the Windows Python distribution and has a Python Shell (right click, open in IDLE, F5).

If you really want to make it work in the cmd:

You have to set Lucida Console as font in cmd. Then:

> chcp
Active code page: 850
> chcp 65001

Then you should have unicode output in the cmd. Your “Active code page” might be different. Note that somewhere, because you might want to change it back afterwards:

> chcp 850

Otherwise you will run into other problems (starting .bat files doesn’t work).
(See also batch-file-encoding)

In your script you also need this:

import codecs

def cp65001(name):
    """This might be buggy, but better than just a LookupError
    """
    if name.lower() == "cp65001":
        return codecs.lookup("utf-8")

codecs.register(cp65001)

Otherwise python will crash. (see windows-cmd-encoding-change-causes-python-crash)

I had a similar bug report for my script.

---

You might also consider using a library to access the MusicBrainz Web Service.
Python-musicbrainzngs works with the current ws/2.