Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
I’ve this hadoop project that someone else coded (link). I’ve the source. I want to implement this on my cluster (basically 3 ubuntu machines). But the mention project works on an EC2 platform (with Cloudera distribution).
So, what all should I install on my systems to make it have the software for running such a project?
I thought about Cloudera Manager, Oracle Java.
If the project works with cloudera distribution (not with EMR), you can install cloudera and it should be fine. Only corner I can expect as problematic – if s3 was used as a file system.
If the project indeed works against s3 you have two ways:
a) Try to replace s3 to hdfs and all file names / paths, and it should also work fine (if they are hardcoded).
b) Install OpenStack’s Swift which is open source alternative to S3 and then try to run Hadoop over it. Disclosure: I am involved in project of running hadoop over Swift. https://github.com/Dazo-org/swift