Question

I am trying to do 2 things: get a return value from a C dll function, and have that same function modify 1 of the members of a structure that is passed to it. After much experimentation I was able to get the function to return a value, but I am still unable to get it to return a modified value to the C# code; the value remains (unmodified) as 0.
I’ve tried lots of variations (ref, [In,Out], etc) to no avail

using System;
using System.Drawing;
using System.Linq;
using System.Runtime.InteropServices;

namespace Vexing.Problem{
    public class myIdx : VexingObject {
        public myIdx(object _ctx) : base(_ctx) { }
        private IPlotObject plot1;
    [StructLayout(LayoutKind.Sequential)] 
    public class PLEX { public int yowser; }

    [DllImport("my.dll", CharSet = CharSet.Unicode)]
    public static extern int cFunction(
               [MarshalAs(UnmanagedType.LPStruct)] PLEX mPlex);

    PLEX a;
    protected override void Create() { a = new PLEX(); }
    protected override void CalcBar() {
        int mf = cFunction(a);
        plot1.Set(a.yowser); }
}}

// pertinent c dll code
typedef struct s_plex { int yowser;} cplex;

extern "C" __declspec( dllexport )  
int cFunction(cplex *Cplex){ Cplex->yowser = 44; return 1;}

Answer 1

Your import declaration is wrong.
Setting the CharSet in your case doesn’t make any sense (there are no string parameters in the native function declaration).
If you want to pass class instance, ref/out also must be thrown away (classes being passed by reference).
And the main point: extern "C" __declspec( dllexport ) means CallingConvention.Cdecl.

UPDATE. Here’s complete working code sample:
C++ (header):

struct CStruct
{
    int myField;
};

extern "C" __declspec( dllexport ) int MyFunction(CStruct *pStruct);

C++ (code):

int MyFunction(CStruct *pStruct)
{
    pStruct->myField = 100;
    return 1;
}

C#:

[StructLayout(LayoutKind.Sequential)]
class MyStruct
{
    public int myField;
};

class Program
{
    MyStruct myStruct = new MyStruct();

    [DllImport("MyLib.dll", CallingConvention = CallingConvention.Cdecl)]
    static extern int MyFunction(MyStruct pStruct);

    static void Main(string[] args)
    {
        var p = new Program();
        var result = MyFunction(p.myStruct);

        Console.WriteLine("Result: {0}, MyStruct.myField = {1}", result, p.myStruct.myField);
    }
}

Prints:

Result: 1, MyStruct.myField = 100