Question

I’m working on research model which outputs results to matlab’s .mat file format, and initially linked with the matlab library to use its file outputting functions.
Recently, requirements changed (who’d have guessed) and the previously linux-only code now has to be compiled on windows, and preferably not require matlab for building -but still output .mat files.
So I searched and found libmatio (http://sourceforge.net/projects/matio/). Although this is easy to link with in linux (you just install it from repositories), its terrible on windows (there is basically no information on building it on windows). In fact it seems windows support was actually silently dropped in version 1.3.3 (back in 2008).
Also, the APi is completely different from the one provided by matlab, which would require me to rewrite/restructure way to much code.

So i came up with this crazy idea…
I needed a drop-in replacement for the Matlab API, preferably without using a library (to make it easy to compile for non-programmers), so i started writing one.
I’m only implementing the functionality I need (writing arrays of doubles, strings, and complex doubles, as well as structures, and structure nesting). All of those already work fine, except for one: arrays of structures.

All matlab data is contained in one struct called ‘mxArray’, and depending on it’s type, it contains pointers to double, complex double, or one or more other mxArray.
The final step just before writing an mxArray to a file is calculating it’s size (and that of it’s children) in Bytes, by caling calcArraySize().
That causes a segfault at some point, because I’m tryng to access a null pointer. To track down the cause I ran the code through valgrind. As always, I try to take care of any issues in the order they arise, as they may be the cause of what’s happening later.
So the first thing valgrind tells me about is:

==8405== Invalid write of size 8
==8405==    at 0x00404541: mxSetFieldByNumber (mxSetFieldByNumber.c:18) [A]
==8405==    by 0x00411679: calcAllRayInfo (calcAllRayInfo.c:156)
==8405==    by 0x0041dd42: main (cTraceo.c:111)
==8405==    Address 0x5500250 is 0 bytes inside a block of size 4 alloc'd
==8405==    at 0x04c28f9f: malloc (vg_replace_malloc.c:236)
==8405==    by 0x00401066: mallocChar (toolsMemory.c:69)
==8405==    by 0x00404314: mxCreateStructMatrix (mxCreateStructMatrix.c:43) [B]
==8405==    by 0x00411235: calcAllRayInfo (calcAllRayInfo.c:105)
==8405==    by 0x0041dd42: main (cTraceo.c:111)

NOTE: I marked [A] and [B] in the code below.
The Structure definition (showing only relevant members):

struct mxArray{
  bool           isStruct;  //determines if this mxArray is a structure (which contains other mxArrays)
  bool           isChild;   //determines wheter this mxArray is a Child of another (when set, its name will not be written to the matfile, as it is already defined in the parent's fieldnames
  uintptr_t      nFields;
  char           **fieldNames;  //something like: {"theta","r","z"};
  struct mxArray **field; //pointer to member mxArrays. only used when isStruct is set.
};
typedef struct mxArray mxArray;

The function I use to allocate memory for a structMatrix along with it’s contents:

mxArray* mxCreateStructMatrix(uintptr_t nRows, uintptr_t nCols, uintptr_t nFields, const char **fieldNames){
  /*
   * creates a 2D array of structures
   */
  mxArray*  outArray = NULL;

  /* do some input value validation */

  // allocate memory
  outArray  = malloc(nRows*nCols*sizeof(mxArray));
  if (outArray == NULL){
    fatal("mxCreateStructMatrix(): memory allocation error.");
  }

  // allocate memory for structure members (fields)
  for (uintptr_t iStruct=0; iStruct<nCols*nRows; iStruct++){
    outArray[iStruct].nFields       = nFields;
    outArray[iStruct].fieldNames        = malloc(nFields*sizeof(char*));

    //copy fieldnames into struct info
    for (uintptr_t iField=0; iField<nFields; iField++){
      //NOTE: strlen returns length of string not including the terminating NULL character
      outArray[iStruct].fieldNames[iField] = mallocChar(strlen(fieldNames[iField])+1);  // [B] <=======
      strncpy(outArray[iStruct].fieldNames[iField], fieldNames[iField], strlen(fieldNames[iField]));
    }

    outArray[iStruct].field     = NULL;
    outArray[iStruct].field     = malloc(nFields*sizeof(mxArray*));
    if (outArray[iStruct].field == NULL){
      fatal("mxCreateStructMatrix(): memory allocation error.\n");
    }
  }
return outArray;
}

Two other allocation functions exist for mxArrays:

mxArray* mxCreateDoubleMatrix(uintptr_t nRows, uintptr_t nCols, uintptr_t numericType){
  /*
   * creates a 2D array of double precision floating point values.
   * can be real or complex.
   */
  [snip]
}
mxArray* mxCreateString(const char *inString)
  /*
   * creates an mxArray containing a string.
   */
  [snip]
}

This functions assigns one mxArray to be the child of another:

void    mxSetFieldByNumber(mxArray* mxStruct,       //pointer to the mxStruct
                           uint32_t index,      //linear index of the element 
                           uint32_t iField,     //index of the structure's field which we want to set.
                           mxArray* inArray){       //the mxArray we want to assign to the mxStruct
  /* 
   * Assigns an mxArray to one of the fields of a structArray
   */
  inArray->isChild = true;  //determines that this mxArray is a child of another one
  mxStruct[index].field[iField] = inArray;  // [A] <===============
}

Usage is:

//create parent mxArray:
mxStruct = mxCreateStructMatrix(1, //number of rows
                                1, //number of columns
                                2, //number of fields in each element
                                fieldNames1);   //list of field names

//create children:
mxY = mxCreateDoubleMatrix(1 ,1, mxREAL);
mxZ = mxCreateDoubleMatrix(1 ,1, mxREAL);
mxSubStruct = mxCreateStructMatrix(1, //number of rows
                                   1, //number of columns
                                   3, //number of fields in each element
                                   fieldNames2); //list of field names

/* copy some values into the mxArrays */
[snip]

//link children to parents
mxSetFieldByNumber( mxStruct, //pointer to the parent mxArray
                    0,        //index of the element (linear)
                    0,        //position of the field (in this case, field 0 is "w"
                    mxY);     //the mxArray we want to add to the mxStruct

mxSetFieldByNumber( mxStruct,   0,  1,  mxZ);

mxSetFieldByNumber( mxSubStruct,    0,  0,  mxY);
mxSetFieldByNumber( mxSubStruct,    0,  1,  mxZ);

mxSetFieldByNumber( mxStruct,   0,  2,  mxSubStruct);

So aparently, mxStruct[index].field[iField] = inArray; is writing into mxStruct[index].fieldNames, thus leaving mxStruct[index].field[iField] == NULL, which then causes a segfault when i try to access it.
How can this be? both are allocated correctly when calling mxCreateStructMatrix, so how can these pointers overlap? What is it I’m overlooking?

Answer 1

I think the problem is with your very last statement:

mxSetFieldByNumber( mxStruct,   0,  /* THIRD FIELD */ 3,  mxSubStruct);

you are trying to assign the third field of mxStruct another nested structure variable, the problem is mxStruct was defined with only two fields:

mxStruct = mxCreateStructMatrix(1, 1, /* TWO */ 2, fieldNames1);

Unlike MATLAB, your code (as far as I can tell) does not support adding structure fields on the fly:

%# -- MATLAB code --
s = struct('f1',[], 'f2',[]);
s.f3 = 99;       %# add a new field

This wouldn’t be very hard to implement, you simply re-allocate the pointer arrays to accommodate one more field and increment the field count.