I have an existing Avro RPC client that sends data to an Avro RPC server. The Avro RPC server currently writes the data into HDFS (and does other things as well). We are changing our server processes to be based upon Storm. I am hoping to find an easy way to get my data into Storm, hopefully using the Avro RPC messages I have now.
I have been looking around for a way to do this, so far with no success. Storm has an RPC model, but it appears to be limited to passing strings, which I want to avoid (why I went to Avro in the first place). Zeromq might be a possibility, but seems limited for what I am trying to do.
Can someone suggest an elegant way for me to get my Avro RPC, schema-based data, into Storm??
Thanks!!!!
so… have not figured out a way to do this directly… but the solution we came up with was a Storm call back procedure that would ask Avro RPC for data. So, basically we switched the client/server relationship. Seems to work well.