It seems like there isn’t any way to append data to an existing Avro serialized file. I’d like to have multiple processes writing to a single avro file, but it looks like each time I open it, I start over from scratch. I don’t want to read in all the data and then write it back out again.
Using the ruby example code I have tried “ab” and “ab+” as various settings, but no joy.
file = File.open('data.avr', 'wb')
schema = Avro::Schema.parse(SCHEMA)
writer = Avro::IO::DatumWriter.new(schema)
dw = Avro::DataFile::Writer.new(file, writer, schema)
dw << {"username" => "john", "age" => 25, "verified" => true}
dw << {"username" => "ryan", "age" => 23, "verified" => false}
dw.close
don’t pass the schema to append to the file