Question

I am wondering if there are any task examples which are not adequate for MapReduce. Some concrete examples would be great to have. thanks!

Answer 1

Some of these might be subjective, so I’ll try to stick with the most agreed upon ones:

• Streaming data — MapReduce by nature is batch. You have something you write in MapReduce that runs every now and then, but it’s hard to get it so it processes streams of data coming in real time. There are some projects out there that are planning on tackling this, like Storm.
• Real time analytics — Again, MapReduce is batch and is tailored for aggregate and large-scale performance, not getting back answers fast. There is a 1-2 minute overhead to be expected with every job, so getting response times of less than a minute is very challenging with MapReduce. This has a lot to do with Hadoop as a piece of software, rather than MapReduce as a paradigm.
• Record fetching — In the same way that you can’t get analytics to run in a short amount of time, fetching individual records or small pieces of data is not something that is easy to do with MapReduce. MapReduce is good at full-table scans, not picking out a record or two. This is the niche that HBase fills.
• Large-scale reorganization — Not sure what a good term for this would be, but anything that requires a large amount of data to be shuffled over the network can be challenging for your network infrastructure. This is because these problems do not fully utilize data locality. Some examples of this are total order sorting and joins across large data sets.