Question

I’ve run into a situation where I have binary data (created using os.urandom(BLOCK_SIZE)) that I need to store in a Postgres database using Django Models.

I’ve read several references to how Django doesn’t yet support Bytea (VARBINARY) field types yet. I found a reference here but don’t know where to get that library (django_postgresql).

I’m currently using Python 2.7 and Django 1.4 on Ubuntu 12.04 – if that helps.

I’ve read some suggestions as a way around this is to convert my data to hex before storing it.

import binascii
key = binascii.hexlify(value)

Ok, now what? I tried to save this value to the database but get errors:

invalid byte sequence for encoding "UTF8": 0xd6c2

Ok, which data field type should I use to do this?

key = models.TextField(max_length = 200)?

key = models.CharField(max_length = 200)?

key = models.???(max_length = 200)???

I’d like to stay away from creating my own custom field type. First, I’m pretty new to Django and wouldn’t feel comfortable doing this yet. Second, I’m pretty new to databases so I’m not sure where to start on defining something like this.

Any suggestions?

Answer 1

If you get unicode errors when you save the key value, you’re doing it wrong. binascii.hexlify() returns a simple string of hex characters, which will not give you the “invalid byte sequence”.

For more efficient storage, use base64 encoding:

import base64
mymodelinstance.key = base64.b64encode(raw_key_value)

And to decode:

raw_key_value = base64.b64decode(mymodelinstance.key)

You can just use a CharField of length BLOCK_SIZE * 2 to hold hex data, or 4 * ( BLOCK_SIZE / 3) for base64.