Question

I know that Clojure works well on a multicore machine, but I was wondering if it would make sense for it to work over a JVM cluster spread out over many machines?

Answer 1

Runa looked into using Terracotta for this and ended up publishing swarmiji as a distributed agent library.

One of the real differences between an SMP system and a Cluster is shared memory. In a Cluster, code has to ask for data, whereas in SMP it can just read it directly. This has some nice advantages and some (scaling) disadvantages.

Clojure’s STM, which differes quite significantly from the many other STM systems out there, is built upon the notion of relative time as measured by a generation counter per transaction. Without common access to this generation counter it cannot give events an order and can’t do it’s job (please forgive this overly simple explanation).

One of the STM’s main motivations was to create a system that really took advantage of shared memory concurrency by ensuring, for instance, that readers never wait for writers and readers always see valid data. Because this was build to take advantage of shared memory it loses a lot of its appeal without shared memory.

The actor model (ala Erlang) is a better fit for distributed computing.

Or, in other words: perhaps we should not try to put a square peg in a distributed concurrent hole.