I tried to find out over the google but no good reference that I found
- I have a Quad-core Ubuntu box running a map-reduce job.
- running default 2 maps and taking lot of time
- what be a good number of map/reduce job for a machine of such config?
Please advice
TIA
Seems like an empirical problem. Start at 1 and work your way up for awhile. Plot a curve, see how it ends up. I’d expect you’ll probably find the sweet spot at either the number of cores available or the number of hardware threads available (may be the same on your box). However, doing the actual experiment is the best way to find out.