In a test I’m building here my goal is to create a parser. So I’ve built a concept proof that reads all messages from a file, and after pushing all of them to memory I’m spawning one process to parse each message. Until that, everything is fine, and I’ve got some nice results. But I could see that the erlang VM is not using all my processor power (I have a quad core), in fact it is using about 25% percent of my processor when doing my test. I’ve made a counter-test using c++ that uses four threads and obviously it is using 100% thus producing a better result (I’ve respected the same queue model erlang uses).
So I’m wondering what could be “slowing” my erlang test? I know it’s not a serialization matter as I’m spawning one process per message. One thing I’ve thought is that maybe my message is too small (about 10k each), and so making that much of processes is not helping achieve a great performance.
Some facts about the test:
106k messages
On erlang (25% processor power used) – 204 msecs
On my C++ test (100% processor power used) – 80 msecs
Yes the difference isn’t that great but if there is more power available certainly there is more room for improvement, right?
Ah, I’ve done some profilling and wasn’t able to find another way to optimize, since there are few function calls and most of them are string to object convertion.
Update:
Woooow! Following Hassan Syed idea, I’ve managed to achieve 35 msecs against 80 from c++! This is awesome!
It seems your erlang VM is using only one core.
Try starting it like this:
The -smp enable flag tells Erlang to start the runtime system with SMP support enabled
With +S 4 you start 4 Erlang schedulers (1 for each core)
You can see if you have SMP enabled when you start the shell:
[smp:2:2] tells it is running with smp enabled 2 schedulers 2 schesulers online