Question

I can get this to work:

[<DllImport("user32.dll")>]
extern bool GetClientRect(nativeint, RECT*)

let getClientRect hwnd =
    let mutable r = RECT()
    if GetClientRect(hwnd, &&r) = false then
        raise <| System.Exception("GetClientRect failed")
    r

But for whatever reason, this just leaves r zero’d, with no exception thrown:

[<DllImport("user32.dll")>]
extern bool GetClientRect(nativeint, [<Out>] RECT rect)

let getClientRect hwnd =
    let mutable r = RECT()
    if GetClientRect(hwnd, r) = false then
        raise <| System.Exception("GetClientRect failed")
    r

Of course, the problem with using pointers is that I get the warning warning FS0051: The use of native pointers may result in unverifiable .NET IL code, and understandably so.

What could I be missing?

Edit: Even though it’s been answered already, for the record, the struct definition:

[<Struct>]
type RECT =
    val left:int
    val top:int
    val right:int
    val bottom:int

Answer 1

It’s not very well documented, but using the ampersand (&) symbol in an F# P/Invoke signature is the correct (and best) way to pass a value by reference. It compiles to the same type signature as a C# ref parameter; adding [<Out>] to the parameter gives the same signature as a C# out parameter.

[<DllImport("user32.dll")>]
extern bool GetClientRect(nativeint hWnd, [<Out>] RECT& rect)

let getClientRect hwnd =
    let mutable r : RECT = Unchecked.defaultOf<_>
    if GetClientRect(hwnd, &r) = false then
        raise <| System.Exception("GetClientRect failed")
    r

Notice that once you change the P/Invoke signature to use byref<RECT> instead of nativeptr<RECT>, you also need to change the address-of operator (&&) you used on r to a single ampersand (&).

Since you’re only getting a value out of GetClientRect, it’s a good practice to initialize the mutable result value to Unchecked.defaultOf<_> so it’s clear that it’s meant to be overwritten by GetClientRect.

EDIT: If changing the P/Invoke signature to use byref (&) doesn’t work, you could also try explicitly specifying the marshalling behavior on the return type. The MSDN documentation for GetClientRect says it returns a BOOL, which is not quite the same as a .NET bool (BOOL is a 4-byte integer value).

If you want to try that:

[<DllImport("user32.dll")>]
extern [<return: MarshalAs(UnmanagedType.Bool)>] bool GetClientRect(
    nativeint hWnd, [<Out>] RECT& rect)