Question

New Library: XParsec

This question has lead to a stream-type-independent parsec implementation in F# 3.0 – inspired by FParsec, freed from CharStreams and simplified: http://corsis.github.com/XParsec/

---

In an FParsec-inspired stream-type-independent simple parsec implementation, I wonder how I could distinguish the following at the type level:

• parsers that consume a piece of stream
• parsers that work on the current position without moving ahead in the stream

Specifically, how can I restrict in F#

• many1?
• skipMany1?'?

to work only with parsers that are type-declared to consume streams?

Does F# offer a similar construct to Haskell’s newtype?

Is there a more F#-specific way to solve this problem?

Code

// Copyright (c) Cetin Sert 2012
// License: Simplified BSD.

#if INTERACTIVE
#else
module XParsec
#endif

  open System
  open System.Collections.Generic

  module Streams =

    type 'a ArrayEnumerator (a : 'a [], ?i : int) as e =
      let         l = a.Length
      let mutable s = -1 |> defaultArg i
      member e.Current           = a.[s]
      member e.Reset          () = s <- -1 |> defaultArg i
      member e.MoveNext       () = let i = s + 1 in if i <  l then s <- i; true else false
      member e.MoveBack       () = let i = s - 1 in if i > -1 then s <- i; true else false
      member e.State with get () = s and   set i =  if i <  l then s <- i       else raise <| ArgumentOutOfRangeException()
      member e.Copy           ()           = new ArrayEnumerator<_>(a, s)
      static member inline New (a : 'a []) = new ArrayEnumerator<_>(a)
      interface 'a IEnumerator with
        member i.Current     = e.Current
      interface Collections.IEnumerator with
        member i.Current     = e.Current :> obj
        member i.MoveNext () = e.MoveNext ()
        member i.Reset    () = e.Reset    ()
      interface IDisposable with
        member i.Dispose  () = ()

    type 'a IEnumerator with
      member inline e.Copy     () = (e :?> 'a ArrayEnumerator).Copy     ()
      member inline e.MoveBack () = (e :?> 'a ArrayEnumerator).MoveBack ()

    type 'a  E = 'a     IEnumerator
    type 'a AE = 'a ArrayEnumerator
    type 'a  S = 'a      E

  open Streams

  type 'a Reply      = S of 'a | F
  type 'a Reply with
    member inline r.Value   = match r with S x -> x | F -> raise <| new InvalidOperationException()
    member inline r.IsMatch = match r with F -> false | S _ -> true 
    static member inline FromBool b = if b then S () else F
    static member inline Negate   r = match r with F -> S () | S _ -> F
    static member inline Map    f r = match r with F -> F    | S x -> S <| f x
    static member inline Put    x r = match r with F -> F    | S _ -> S x
    static member inline Choose f r = match r with F -> F    | S x -> match f x with Some v -> S v | None -> F

  type 'a R = 'a Reply

  type Parser<'a,'b> = 'a S -> 'b R

  module Primitives =

    open Operators

    let inline attempt (p : Parser<_,_>) (s : _ S) = s.Copy() |> p

    let inline Δ<'a> = Unchecked.defaultof<'a>
    let inline pzero     (_ : _ S) = S Δ
    let inline preturn x (_ : _ S) = S x

    let inline current   (e : _ S) = e.Current |> S
    let inline one       (e : _ S) = if e.MoveNext() then e |> current else F

    let inline (?->) b x = if b then Some x else None
    let inline (!!>) (p : Parser<_,_>)   e = e |> p |> Reply<_>.Negate
    let inline (|->) (p : Parser<_,_>) f e = e |> p |> Reply<_>.Map    f
    let inline (|?>) (p : Parser<_,_>) f e = e |> p |> Reply<_>.Choose f
    let inline (>.)  (p : Parser<_,_>) (q : Parser<_,_>) e = match p e with F -> F   | S _ -> q e
    let inline (.>)  (p : Parser<_,_>) (q : Parser<_,_>) e = match p e with F -> F   | S p -> q e |> Reply<_>.Put p
    let inline (.>.) (p : Parser<_,_>) (q : Parser<_,_>) e = match p e with F -> F   | S p -> q e |> Reply<_>.Map (fun q -> (p,q))
    let inline (</>) (p : Parser<_,_>) (q : Parser<_,_>) e = match p e with F -> q e | s   -> s

    let inline private back              (s : _ S) = s.MoveBack() |> ignore
    let inline many    (p : Parser<_,_>) (s : _ S) = let r = ref Δ in let q = Seq.toList <| seq { while (r := p s; (!r).IsMatch) do yield (!r).Value } in back s; S q
    let inline many1   (p : Parser<_,_>) (s : _ S) = s |> many p |> Reply<_>.Choose (function _::_ as l -> Some l | _ -> None)
    let inline array n (p : Parser<_,_>) (s : _ S) = s |> many p |> Reply<_>.Choose (function l -> let a = l |> List.toArray in (a.Length = n) ?-> a)

    let inline skipMany'  (p : Parser<_,_>) (s : _ S) = let c = ref 0 in (while (p s).IsMatch do c := !c + 1); back s; S !c
    let inline skipMany   (p : Parser<_,_>) (s : _ S) = s |> skipMany'  p |> Reply<_>.Put ()
    let inline skipMany1' (p : Parser<_,_>) (s : _ S) = s |> skipMany'  p |> Reply<_>.Choose (fun n -> if n > 0 then Some n  else None)
    let inline skipMany1  (p : Parser<_,_>) (s : _ S) = s |> skipMany1' p |> Reply<_>.Put ()
    let inline skipN   i   p                 s        = s |> skipMany'  p |> Reply<_>.Choose (fun n -> if n = i then Some () else None)

    let inline (!*) p s = skipMany  p s
    let inline (!+) p s = skipMany1 p s

Answer 1

No, F# does not have anything like newtype.

If you want to declare a new type (that is treated as a different type by the type checker), then you have to define it as a wrapper, for example using single-case discriminated union:

type NewParser = NP of OldParser

Another way to distinguish between multiple varsions of a type is to use phantom types. This is pretty subtle technique and is not used too often (more of a research topic), but I wrote an article about using it with F# async and it’s quite powerful.

The general design principle in F# is to keep things simple, so this may be too much, but here is an example: (BTW: I’d also suggest using fewer operators and more named functions that are easier to understand)

// Interfaces that do not implement anything, just represent different parser kinds
type ParserBehaviour = 
  interface end
type ConstParser = 
  inherit ParserBehaviour
type ForwardParser = 
  inherit ParserBehaviour

In the definition of the parser, you can now add a type parameter that is not used and has to be one of these interfaces:

type Parser<'T, 'F when 'F :> ParserBehaviour> = 
  P of (IEnumerator<char> -> 'T)

Now, you can annotate parsers with their behaviour:

let current : Parser<_, ConstParser> = P (fun c -> c.Current)
let next : Parser<_, ForwardParser> = P (fun c -> c.MoveNext; c.Current)

And if you want to write a function that can only work on parsers that do not change the Ienumerator, you can require Parser<'T, ConstParser>. For functions that can work on all of them, you can take Parser<'T, 'B>.

… but as I said, this is fairly advanced and some would consider this a black magic in F#. The F# approach to programming is quite different than, say, Haskell. It is more important to create simple and easy to use library than to be fully type-safe in every case.