WARNING The code I provide in my question may crash matlab AND your machine!!!
I have written a mex gateway function to a C++ class. If I compile this mex function on 32Bit windows using R2008a I have no problems. If I compile and run on Matlab R2011a running 64bit Scientific Linux (a version of Red Hat Enterprize Linux) matlab exits with a segfault when the mexfunction is called, although it appears to run about halfway through the program. The C++ class can be compiled and run (with a main function) outside of Matlab on both platforms with no errors. I am using Microsoft Visual C++ Express Edition on the windows machine, and gcc 4.4.5 on the Linux machine.
What is the cause of this and can I fix it?
A zip file containing the code and data files necessary to reproduce the problem can be downloaded from http://www.see.ed.ac.uk/~s0237326/downloads/mexcrash.zip. This zip file contains the .m and .cpp source code, and a text file for testing (Temp.fem, 10kB). The file fmeshersetup.m shows the commands I am using to compile. The file Test_mexfmesher.m runs the mexfunction with an appropriate input for testing. The mex gateway function is mexfmesher.cpp, it calls the fmesher class which is made up of the files in the fmesher directory.
There are a total of 13 C++ source files, which is a bit much to expect someone to debug, even the particular line on which the segfault occurs woud be of interest.
Unfortunately I do not have access to a graphical debugger on 64bit linux that can interface with Matlab, and was hoping someone would immediately see the problem, or it would be a known problem. As stated, when run not as a mex file, the program runs fine, so I cannot locate the error in another tool, and I’m not very familiar with gdb. I believe the problem might be related to a calls to the C++ function ‘free’.
Below is a backtrace from the segfault:
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff721be36 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwservices.so
(gdb) backtrace
#0 0x00007ffff721be36 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwservices.so
#1 0x00000034daa914f2 in std::basic_ostream<char, std::char_traits<char> >::flush() () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libstdc++.so.6
#2 0x00007ffff72302b0 in ioFlush () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwservices.so
#3 0x00007ffff61303f5 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#4 0x00007ffff61330ec in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#5 0x00007ffff6130c7a in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#6 0x00007ffff6131741 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#7 0x00007ffff618a7d9 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#8 0x00007ffff686d7ef in Mfh_file::dispatch_fh(int, mxArray_tag**, int, mxArray_tag**) () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_dispatcher.so
#9 0x00007ffff61731f0 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#10 0x00007ffff6114975 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#11 0x00007ffff612e96e in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#12 0x00007ffff61330ec in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#13 0x00007ffff6130c7a in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#14 0x00007ffff6131741 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#15 0x00007ffff618a7d9 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#16 0x00007ffff686d7ef in Mfh_file::dispatch_fh(int, mxArray_tag**, int, mxArray_tag**) () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_dispatcher.so
#17 0x00007ffff61669b2 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#18 0x00007ffff6128e13 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#19 0x00007ffff6127eb7 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#20 0x00007ffff6128397 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#21 0x00007ffff6d378fe in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwbridge.so
#22 0x00007ffff6d384ae in mnParser () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwbridge.so
#23 0x00007ffff6ae0d39 in mcrInstance::mnParser_on_interpreter_thread() () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwmcr.so
#24 0x00007ffff6ac3db2 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwmcr.so
#25 0x00007ffff6ac3ec0 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwmcr.so
#26 0x00007fffee8badb6 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/../../bin/glnxa64/libmwuix.so
#27 0x00007fffee8c413d in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/../../bin/glnxa64/libmwuix.so
#28 0x00007fffef3110bd in sysq::during_F<boost::weak_ptr<sysq::ws_ppeHook>, boost::shared_ptr<sysq::ws_ppeHook> > std::for_each<__gnu_cxx::__normal_iterator<boost::weak_ptr<sysq::ws_ppeHook>*, std::vector<boost::weak_ptr<sysq::ws_ppeHook>, std::allocator<boost::weak_ptr<sysq::ws_ppeHook> > > >, sysq::during_F<boost::weak_ptr<sysq::ws_ppeHook>, boost::shared_ptr<sysq::ws_ppeHook> > >(__gnu_cxx::__normal_iterator<boost::weak_ptr<sysq::ws_ppeHook>*, std::vector<boost::weak_ptr<sysq::ws_ppeHook>, std::allocator<boost::weak_ptr<sysq::ws_ppeHook> > > >, __gnu_cxx::__normal_iterator<boost::weak_ptr<sysq::ws_ppeHook>*, std::vector<boost::weak_ptr<sysq::ws_ppeHook>, std::allocator<boost::weak_ptr<sysq::ws_ppeHook> > > >, sysq::during_F<boost::weak_ptr<sysq::ws_ppeHook>, boost::shared_ptr<sysq::ws_ppeHook> >) ()
from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/../../bin/glnxa64/libuij.so
#29 0x00007fffef312989 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/../../bin/glnxa64/libuij.so
#30 0x00007fffef30f4ae in svWS_ProcessPendingEvents(int, int, bool) () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/../../bin/glnxa64/libuij.so
#31 0x00007ffff6ac21c7 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwmcr.so
#32 0x00007ffff6ac260a in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwmcr.so
#33 0x00007ffff6ac2d6f in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwmcr.so
#34 0x00000034d7a077e1 in start_thread () from /lib64/libpthread.so.0
#35 0x00000034d72e573d in clone () from /lib64/libc.so.6
(gdb) quit
You have attempted to use ‘free’ to free memory with a pointer which actually points to NULL. A careful programmer would check if this is the case before attempting to use free. Some libraries will have an automatic check for this.
The reason you did not notice this when compiling as a standalone may be due to the fact that matlab uses its own libstdc++ (stored in /opt/matlab-2011a/bin/glnxa64/libstdc++.so.6) instead of the main one (stored in /usr/lib/x86_64-linux-gnu/).
If you are sudoer, You should force matlab to use the main libstdc++, this can solve many problems with mex files: