Question

I need to execute MapReduce on my Cassandra cluster, including data locality, ie. each job queries only rows which belong to local Casandra Node where the job runs.

Tutorials exist, on how to setup Hadoop for MR on older Cassandra version (0.7). I cannot find such for current release.

What has changed since 0.7 in this regard ?

What software modules are required for minimal setup (Hadoop+HDFS+…)?

Do I need Cassandra Enterprise ?

Answer 1

Cassandra contains a few classes which are sufficient to integrate with Hadoop:

• ColumnFamilyInputFormat – This is an input for a Map function which can read all rows from a single CF in when using Cassandra’s random partitioner, or it can read a row range when used with Cassandra’s ordered partitioner. Cassandra cluster has ring form, where each ring part is responsible for concrete key range. Main task of Input Format is to divide Map input into data parts which can be processed in parallel – those are called InputSplits. In Cassandra case this is simple – each ring range has one master node, and this means that Input Format will create one InputSplit for each ring element, and it will result in one Map task. Now we would like to execute our Map task on the same host where data is stored. Each InputSplit remembers IP address of its ring part – this is the IP address of Cassandra node responsible to this particular key range. JobTracker will create Map tasks form InputSplits and assign them to TaskTracker for execution. JobTracker will try to find TaskTracker which has the same IP address as InputSplit – basically we have to start TaskTracker on Cassandra host, and this will guarantee data locality.
• ColumnFamilyOutputFormat – this configures context for Reduce function. So that the results can be stored in Cassandra
• Results from all Map functions has to be combined together before they can be passed to reduce function – this is called shuffle. It uses local file system – from Cassandra perspective nothing has to be done here, we just need to configure path to local temp directory. Also there is no need to replace this solution with something else (like persisting in Cassandra) – this data does not have to be replicated, Map tasks are idempotent.

Basically using provided Hadoop integration gives up possibility to execute Map job on hosts where data resides, and Reduce function can store results back into Cassandra – it’s all that I need.

There are two possibilities to execute Map-Reduce:

• org.apache.hadoop.mapreduce.Job – this class simulates Hadoop in one process. It executes Map-Resuce task and does not require any additional services/dependencies, it needs only access to temp directory to store results from map job for shuffle. Basically we have to call few setters on Job class, which contain things like class names for Map task, Reduce task, input format, Cassandra connection, when setup is done job.waitForCompletion(true) has to be called – it starts Map-Reduce task and waits for results. This solution can be used to quickly get into Hadoop world, and for testing. It will not scale (single process), and it will fetch data over network, but still – it will be fine for beginning.
• Real Hadoop cluster – I did not set it up yet, but as I understood, Map-Reduce jobs from previous example will work just fine. We need additionally HDFS which will be used to distribute jars containing Map-Reduce classes in Hadoop cluster.