Question

I recently joined a company which is using django to build their product. I’m currently responsible for one of the apps, which was already developed a little bit before I was here.

One of the entities in the app has a json dictionary attribute, which has been kept in the db as a text field. Also, this attribute is marked in the model as a text field. So, as you can imagine it’s not being handled correctly.

I wanted to change this and set it as a json field using https://github.com/bradjasper/django-jsonfield , which works really well.

However, I’ve run into a peculiar problem. Previous data stored in the db was not correctly handled and since it was unicode data, the text field in the db looks like:

{u'key': u'value'}

Now when the entity manager tries to load those values using the json field, it of course breaks since it’s no longer a valid json string.

I’ve done some research on how to overcome this, but haven’t found nothing.

My question:

Do you have any suggestion on how to overcome this? It can be any type of solution.

• Something that I can run over night altering that field to transform it to a valid json string.
• Some changes to the json-field code, which enables it to correctly handle these values.

Additional info

We use postgres with psycopg2 as django’s db backend.

Thank you very much.

Answer 1

You’re probably just going to need to iterate over the whole table, load the field, convert it into a real Python dict, and dump it back out with json.dumps. ast.literal_eval is a good choice for the conversion stage because it works like the built-in eval but is more restricted, so less risky to your system.

for obj in MyModel.objects.all():
    value = ast.literal_eval(obj.dict_value)
    obj.dict_value = json.dumps(value)
    value.save()