Question

Im trying to achieve something similar to the first example under “Sending large amounts of data” at http://www.playframework.org/documentation/2.0/ScalaStream.

What I am doing differently is that I have more than one file that I want to concatenate in the response. It looks like this:

def index = Action {

  val file1 = new java.io.File("/tmp/fileToServe1.pdf")
  val fileContent1: Enumerator[Array[Byte]] = Enumerator.fromFile(file1)    

  val file2 = new java.io.File("/tmp/fileToServe2.pdf")
  val fileContent2: Enumerator[Array[Byte]] = Enumerator.fromFile(file2)   

  SimpleResult(
    header = ResponseHeader(200),
    body = fileContent1 andThen fileContent2
  )

}

What happens is that only the contents of the first file is in the response.

Something slightly simpler like below works fine though:

fileContent1 = Enumerator("blah" getBytes)
fileContent2 = Enumerator("more blah" getBytes)

SimpleResult(
  header = ResponseHeader(200),
  body = fileContent1 andThen fileContent2
)

What am I getting wrong?

Answer 1

I’ve got most of this from reading through the play code so I might have misunderstood, but from what I can see: the Enumerator.fromFile function(Iteratee.scala [docs][src]) creates an enumerator which when applied to an Iteratee[Byte, Byte] appends an Input.EOF to the output of the Iteratee when it’s done reading from the file.

According to the Play! docs example that you linked to you’re supposed to manually append an Input.EOF (alias of Enumerator.eof) at the end of the enumerator. I would think that, having an Input.EOF automatically appended to the end of the file’s byte array is the reason that you’re getting only one file returned.

The equivalent simpler example in the play console would be:

scala> val fileContent1 = Enumerator("blah") andThen Enumerator.eof
fileContent1: play.api.libs.iteratee.Enumerator[java.lang.String] = play.api.libs.iteratee.Enumerator$$anon$23@2256deba

scala> val fileContent2 = Enumerator(" and more blah") andThen Enumerator.eof
fileContent2: play.api.libs.iteratee.Enumerator[java.lang.String] = play.api.libs.iteratee.Enumerator$$anon$23@7ddeef8a

scala> val it: Iteratee[String, String] = Iteratee.consume[String]()
it: play.api.libs.iteratee.Iteratee[String,String] = play.api.libs.iteratee.Iteratee$$anon$18@6fc6ce97

scala> Iteratee.flatten((fileContent1 andThen fileContent2) |>> it).run.value.get
res9: String = blah

The fix, although I haven’t tried this yet would be to go a few levels deeper and use the Enumerator.fromCallback function directly and pass it a custom retriever function that keeps returning Array[Byte]s until all the files that you want to concatenate have been read. Refer to the implementation of the fromStream function to see what the default is and how to modify it.

An example of how to do this (adapted from Enumerator.fromStream):

def fromFiles(files: List[java.io.File], chunkSize: Int = 1024 * 8): Enumerator[Array[Byte]] = {
  val inputs = files.map { new java.io.FileInputStream(_) }
  var inputsLeftToRead = inputs
  Enumerator.fromCallback(() => {
    def promiseOfChunkReadFromFile(inputs: List[java.io.FileInputStream]): Promise[Option[Array[Byte]]] = {
      val buffer = new Array[Byte](chunkSize)
      (inputs.head.read(buffer), inputs.tail.headOption) match {
        case (-1, Some(_)) => {
          inputsLeftToRead = inputs.tail
          promiseOfChunkReadFromFile(inputsLeftToRead)
        }
        case (-1, None) => Promise.pure(None)
        case (read, _) => {
          val input = new Array[Byte](read)
          System.arraycopy(buffer, 0, input, 0, read)
          Promise.pure(Some(input))
        }
      }
    }
    promiseOfChunkReadFromFile(inputs)
  }, {() => inputs.foreach(_.close())})
}