Question

I use import avro and etc. for compiling avro schemas in python. Sometimes, my avro schema will look like:

{ name: "Message1" ..... }

{ name: "Message2", "fields": [ { "type": "Message1", "name": "previous_avro_schema" } ] } ...

Please ignore the typos. I just want to get the message across. The gist is that I have two avro schemas. One of the avro schema use the second avro schema as one of its fields. How do I call avro.schema.parse(....) for such avro messages so that they can be compiled correctly?

Answer 1

Avro’s Python support is pretty pathetic, but you can get some stuff to work including multiple schemas at once. You just need to merge the schema files together into one file, and you need to make sure that they are merged in the correct order so dependencies come first, and you need to replace the names with the actual schema This is the script that I use to merge them:

def resolve(path):
    "fully resolve a schema that includes other schemas"
    data = open(path).read()
    # fill in any while they remain
    while True:
        beg = data.find('`')
        end = data.find('`', beg + 1)
        if beg < 0:
            break
        path = os.path.join(os.path.dirname(path), data[beg+1:end] + '.avsc')
        data = data[:beg] + resolve(path) + data[end+1:]
    return data