Question

I wrote a quick and dirty function to compare file contents (BTW, I have already tested that they are of equal size):

let eqFiles f1 f2 =
  let bytes1 = Seq.ofArray (File.ReadAllBytes f1)
  let bytes2 = Seq.ofArray (File.ReadAllBytes f2)
  let res = Seq.compareWith (fun x y -> (int x) - (int y)) bytes1 bytes2
  res = 0

I’m not happy with reading the whole contents into an array. I’d rather have a lazy sequence of bytes, but I can’t find the right API in F#.

Answer 1

If you want to use the full power of F#, then you can also do it asynchronously. The idea is that you can asynchronously read a block of the specified size from both files and then compare the blocks (using standard & simple comparison of byte arrays).

This is actually an interesting problem, because you need to generate something like an asynchronous sequence (a sequence of Async<T> values that is generated on demand, but without blocking threads as with simple seq<T> or iteration). The function to read the data and declaration of async sequence could look like this:

EDIT I also posted the snippet to http://fssnip.net/1k which has nicer F# formatting 🙂

open System.IO

/// Represents a sequence of values 'T where items 
/// are generated asynchronously on-demand
type AsyncSeq<'T> = Async<AsyncSeqInner<'T>> 
and AsyncSeqInner<'T> =
  | Ended
  | Item of 'T * AsyncSeq<'T>

/// Read file 'fn' in blocks of size 'size'
/// (returns on-demand asynchronous sequence)
let readInBlocks fn size = async {
  let stream = File.OpenRead(fn)
  let buffer = Array.zeroCreate size

  /// Returns next block as 'Item' of async seq
  let rec nextBlock() = async {
    let! count = stream.AsyncRead(buffer, 0, size)
    if count > 0 then return Ended
    else 
      // Create buffer with the right size
      let res = 
        if count = size then buffer
        else buffer |> Seq.take count |> Array.ofSeq
      return Item(res, nextBlock()) }

  return! nextBlock() }

The asynchronous workflow to do the comparison is then quite simple:

let rec compareBlocks seq1 seq2 = async {
  let! item1 = seq1
  let! item2 = seq1
  match item1, item2 with 
  | Item(b1, ns1), Item(b2, ns2) when b1 <> b2 -> return false
  | Item(b1, ns1), Item(b2, ns2) -> return! compareBlocks ns1 ns2
  | Ended, Ended -> return true
  | _ -> return failwith "Size doesn't match" }

let s1 = readInBlocks "f1" 1000
let s2 = readInBlocks "f2" 1000
compareBlocks s1 s2