Question

I am trying to use following code to retrieve character index by position in a RichTextBox. I know I can use GetCharIndexFromPosition method provided by RichTextBox Class, but I want to know what is wrong with following code:

SendMessage import is this:

[DllImport("User32.dll", EntryPoint = "SendMessage", CharSet= CharSet.Auto)]
public static extern IntPtr SendMessage(IntPtr hWnd, int msg, int wParam, ref POINTL lParam);

And then this call:

int returnVal = (int)WinUser.SendMessage(this._textBox.Handle, (int)WinUser.Message.EM_CHARFROMPOS, 0, ref p);

where p is the POINTL stucture instance containing the screen coordinates with upper-left corner of RichTextBox as origin.

The POINTL Structure is defined as

[StructLayout(LayoutKind.Sequential)]
public struct POINTL
{
    public long x;
    public long y;
}

The POINTL p has been initialized as:

WinUser.POINTL p;
p.x = 0;
p.y = 0;

NOW THE PROBLEM:

If p is initialized as given above the returnVal is 0

If p is anything else like {x = 10, y =10} or {x = 1 and y = 1} the returnVal is 1

In both the cases the function GetCharIndexFromPosition gives the correct index.

Answer 1

Change long to int.
(Win32 LONGs are 32-bit integers that correspond to .Net ints)

.Net’s GetCharIndexFromPosition method is defined as

    public override int GetCharIndexFromPosition(Point pt) { 
        NativeMethods.POINT wpt = new NativeMethods.POINT(pt.X, pt.Y);
        int index = (int)UnsafeNativeMethods.SendMessage(new HandleRef(this, Handle), NativeMethods.EM_CHARFROMPOS, 0, wpt); 

        string t = this.Text;
        // EM_CHARFROMPOS will return an invalid number if the last character in the RichEdit
        // is a newline. 
        //
        if (index >= t.Length) { 
            index = Math.Max(t.Length - 1, 0); 
        }
        return index; 
    }

The NativeMethods.POINT type is defined as

    [StructLayout(LayoutKind.Sequential)] 
    public class POINT {
        public int x; 
        public int y; 

        public POINT() { 
        }

        public POINT(int x, int y) {
            this.x = x; 
            this.y = y;
        } 

#if DEBUG
        public override string ToString() { 
            return "{x=" + x + ", y=" + y + "}";
        }
#endif
    }