I have a couchdb database setup at a AWS EC2 medium on-demand instance, there are around 4 million objects in it, with growing rate of around 100 objects per second.
I want to write some map/reduce queries on top of it, but it takes forever for my map jobs to complete.
SO I am wondering if i should copy over the data to some other machine, and delete all data here on the master machine, keeping it clean, and i should rather write my map jobs on the second instance where the data is copied; i am also thinking of moving over this data to a s3 instance, and keep just one week’s data here.
Am i thinking in right direction
Unfortunately for such a big database you can only use build-in reduce functions:
These functions works MUCH faster than javascript ones. And this is the only possible option for huge databases.
http://wiki.apache.org/couchdb/Built-In_Reduce_Functions