I recently wrote a couple of map/reduce programs in java. But I also know that scripting language like php can also be used. However, mostly everyone recommends java or python. I currently work in php. So I was wondering which language is better suited for map/reduce program development?
One major disadvantage of php for map/reduce implementation is that, it is not multi-threaded.
Also that, hadoop has extensive framework of classes, interfaces and methods specially made in java, which php programs can’t avail. And also that, php isn’t designed to handle heavy data processing task.
So can anyone tell me in broad points which one to choose as a weapon of choice for map/reduce implementation?
shanthanu, your first question is
Q) which scripting language is good for hadoop?
A) Most of the scripting languages like php, python, perl, ruby bash is good. Any language able to read from stdin, write to sdtout and parse tab and new line characters will work: Hadoop Streaming just pipes the string representations of key value pairs as concatenated with a tab to an arbitrary program that must be executable on each task tracker node.
On most linux distros used to setup hadoop clusters, python, bash, ruby, perl… are already installed but nothing will prevent to roll up your own execution environment for your favorite scripting or compiled programming language.
Q) PHP is not multi threaded?
A) yes, but, there are ways through which we can make PHP multi-threaded. For example use: pnctl_fork() (but, this does not work in windows)
The question which you should keep always in mind before going for the scripting languages with hadoop is not “which scripting language?” because anything is ok.
But, the difference between java and scripting language, it is “Heart Beat of child nodes will not be sent to the parent nodes when we are using scripting languages”.