Question

Recent events on the blogosphere have indicated that a possible performance problem with Scala is its use of closures to implement for.

What are the reasons for this design decision, as opposed to a C or Java-style “primitive for” – that is one which will be turned into a simple loop?

(I’m making a distinction between Java’s for and its “foreach” construct here, as the latter involves an implicit Iterator).

More detail, following up from Peter. This bit of Scala:

  object ScratchFor {
    def main(args : Array[String]) : Unit = {
      for (val s <- args) {
        println(s)
      }
    }
  }

creates 3 classes: ScratchFor$$anonfun$main$1.class ScratchFor$.class ScratchFor.class

ScratchFor::main just forwards to the companion object, ScratchFor$.MODULE$::main which spins up an ScratchFor$$anonfun$main$1 (which is an implementation of AbstractFunction1).

It’s in the apply() method of this anonymous inner impl of AbstractFunction1 that the actual code lives, which is effectively the loop body.

I don’t see HotSpot being able to rewrite this into a simple loop. Happy to be proved wrong on this, though.

Answer 1

Traditional for loops are clumsy, verbose and error-prone. I think it is proof enough of this that “for-each” loops where added to Java, C# and C++, but if you want more details you may check item 46 of Effective Java.

Now, for-each loops are still much faster than Scala for-comprehension, but they are also much less powerful (and more clumsy) because they cannot return values. If you want to transform or filter a collection (or do both to a group of collections), you’ll still have to handle all the mechanical details of constructing the result collection in addition to computing the values. Not to mention it inevitably uses some mutable state.

Finally, even though for-each loops are adequate enough for collections, they are not suited to other monadic classes (of which collections are a subset of).

So Scala has a general method which takes care of all of the above. Yes, it is slower, but the goal is to have the compiler effectively optimise it well enough so that this doesn’t become a hindrance (and, of course, JIT could help here as well).

That has not been accomplished to this date, but -optimise has reduced a lot of ground between common for-each loops and for-comprehensions on the latest versions of Scala. If performance is essential, you can always use while or tail recursion.

Now, it would be possibly for Scala to have common for loops or for-each loops as special cases specifically targeted at performance issues (since for-comprehensions can do everything they do). However, that violates two principles that guide Scala’s design:

1. Reduce complexity. Yes, contrary to what some say, that is a design goal, and special cases that serve no other purpose other than optimise performance — even though a workable solution exists for performance cases — would needlessly increase the complexity of the language.

2. Scalability. This is in the sense that the use can scale the language for any size of problem by writing libraries. The point here is that having the compiler optimise one particular class, such as Range, would make it impossible for the user to create a replacement class that would perform just as well.