Question

There is a command line client written in ruby that is used as the standard. However, it doesn’t run in 1.9. There is also a very good aws-sdk for ruby, but it doesn’t support EMR. Is there a good alternative?

Answer 1

UPDATE 6/22/2012: Amazon is reviewing @nkadwa’s pull request (see above). Keep an eye out for that.

I’ve been working with the Ruby command-line client, but have a number of Ruby/Rails based tools I build that require ruby 1.9.3. The workaround for me was RVM.

Let’s say you have $HOME/projects/elastic-mapreduce where you have the EMR Ruby client
Let’s say you also have $HOME/projects/myproj

You can tell your system to use ruby-1.8.7 for EMR with:

echo 'rvm use ruby-1.8.7' > $HOME/projects/elastic-mapreduce/.rvmrc
echo 'rvm use ruby-1.9.3@mygemset' > $HOME/projects/myproj/.rvmrc

If you need to call out to the elastic map-reduce scripts from somewhere, you can rig up RVM as such as the following. By default, elastic-mapreduce uses the credentials.json file in its same directory. Note that the first time you enter the directory, your system will verify that you are willing to trust the .rvmrc file.

# cd'ing into the directory will output stuff to stdout when it changes
cd $HOME/elastic-mapreduce > /dev/null && rvm ruby-1.8.7; ./elastic-mapreduce [args]

Or, if you want to skip the ‘cd’ business, you can do something like:

# Again, redirect output after 'rvm use' so you don't get the green output
system("rvm use ruby-1.8.7 > /dev/null; /path/to/elastic-mapreduce -c /path/to/elastic-mapreduce/credentials.json --list") 

You will need to follow the instructions on the RVM website to get it installed (quite straight forward). But it’s the perfect solution for when you need multiple versions of Ruby running. I have it running in production in EC2.

Hope that helps.