Question

So the Play2.0 Enumeratee page shows an example of using a the &> or through method to change an Enumerator[String] into an Enumerator[Int]:

val toInt: Enumeratee[String,Int] = Enumeratee.map[String]{ s => s.toInt }
val ints: Enumerator[Int] = strings &> toInt

There is also an Enumeratee.grouped enumeratee to create an enumerator of chunks from individual elements. That seemed to work fine.

But what I see is that the usual input would be in the form of Array[Byte] (which is returned by Enumerator.fromFile and Enumerator.fromStream). With that in mind I would like to take those Array[Byte] inputs and turns them into an Enumerator[String], for instance where each string is a line (terminated by a '\n'). The boundaries for the lines and the Array[Byte] elements won’t usually match. How do I write an enumerator that can convert the chunked arrays into chunked strings?

The purpose is to chunk those lines back to the browser as each Array[Byte] becomes available, and keep the leftover bytes that were not part of a complete line until the next input chunk comes along.

Ideally I’d love to have a method that given an iter: Iteratee[Array[Byte], T] and an Enumerator[Array[Byte]] will give me back an Enumerator[T], where my T elements were parsed by iter.

Additional Info: I had a bit of time to clean up my code and here is a specific example of what I’m trying to do. I have the following iteratees that detect the next line:

import play.api.libs.iteratee._
type AB = Array[Byte]

def takeWhile(pred: Byte => Boolean): Iteratee[AB, AB] = {
  def step(e: Input[AB], acc: AB): Iteratee[AB, AB] = e match {
    case Input.EOF => Done(acc, Input.EOF)
    case Input.Empty => Cont(step(_, acc))
    case Input.El(arr) =>
      val (taking, rest) = arr.span(pred)
      if (rest.length > 0) Done(acc ++ taking, Input.El(rest)) 
      else Cont(step(_, acc ++ taking)) 
  }
  Cont(step(_, Array()))
}

val line = for {
  bytes <- takeWhile(b => !(b == '\n' || b == '\r'))
  _     <- takeWhile(b =>   b == '\n' || b == '\r')
} yield bytes

And what I’d like to do is something like that:

Ok.stream(Enumerator.fromFile(filename) &> chunkBy(line)).as("text/plain")

Answer 1

https://github.com/playframework/Play20/commit/f979006a7e2c1c08ca56ee0bae67b5463ee099c1#L3R131 Does something similar to what you are doing. I fixed grouped to take care of the remaining input. The code basically looks like:

val upToNewLine = 
  Traversable.splitOnceAt[String,Char](_ != '\n')  &>>
  Iteratee.consume()

Enumeratee.grouped(upToNewLine)

Also I have to fix repeat in the same way