Question

In Python, I can do something like this:

lazy = ((i,j) for i in range(0,10000) for j in range(0,10000))
sum((1 for i in lazy))

It will take a while, but the memory use is constant.

The same construct in scala:

(for(i<-0 to 10000; j<-i+1 to 10000) yield (i,j)).count((a:(Int,Int)) => true)

After a while, I get a java.lang.OutOfMemoryError, even though it should be evaluated lazily.

Answer 1

Nothing’s inherently lazy about Scala’s for-comprehension; it’s syntactic sugar* which won’t change the fact that the combination of your two ranges will be eager.

If you work with lazy views of your ranges, the result of the comprehension will be lazy too:

scala> for(i<-(0 to 10000).view; j<-(i+1 to 10000).view) yield (i,j)
res0: scala.collection.SeqView[(Int, Int),Seq[_]] = SeqViewN(...)

scala> res0.count((a: (Int, Int)) => true)
res1: Int = 50005000

The laziness here is nothing to do with the for-comprehension, but because when flatMap or map (see below) are called on some type of container, you get back a result in the same type of container. So, the for-comprehension will just preserve the laziness (or lack of) of whatever you put in.

---

*for something like:

(0 to 10000).flatMap(i => (i+1 to 10000).map(j => (i, j)))