Question

I’m developing an application with a team in .Net (C++) and provide a COM interface to interact with python and other languages.

What we’ve found is that pushing data through COM turns out to be pretty slow.

I’ve considered several alternatives:

• dumping data to a file and sending the file path through com
• Shared Memory via mmap?
• Stream data through a socket directly?

From your experience what’s the best way to pass around data?

Answer 1

Staying within the Windows interprocess communication mechanisms, we had positive experience using windows named pipes. Using Windows overlapped IO and the win32pipe module from pywin32.

You can learn much about win32 and python in the Python Programming On Win32 book.

The sending part simply writes to r'\\.\pipe\mypipe'.

A listener (ovpipe) object holds an event handle, and waiting for a message with possible other events involves calling win32event.WaitForMultipleObjects.

rc = win32event.WaitForMultipleObjects(     eventlist,    # Objects to wait for.     0,            # Wait for one object     timeout)      # timeout in milli-seconds. 

Here is part of the python overlapped listener class:

import win32event import pywintypes import win32file import win32pipe  class ovpipe: 'Overlapped I/O named pipe class' def __init__(self):     self.over=pywintypes.OVERLAPPED()     evt=win32event.CreateEvent(None,1,0,None)     self.over.hEvent=evt     self.pname='mypipe'     self.hpipe = win32pipe.CreateNamedPipe(         r'\\.\pipe\mypipe',             # pipe name          win32pipe.PIPE_ACCESS_DUPLEX|   # read/write access         win32file.FILE_FLAG_OVERLAPPED,         win32pipe.PIPE_TYPE_MESSAGE|    # message-type pipe          win32pipe.PIPE_WAIT,            # blocking mode          1,                              # number of instances          512,                            # output buffer size          512,                            # input buffer size          2000,                           # client time-out         None)                           # no security attributes     self.buffer = win32file.AllocateReadBuffer(512)     self.state='noconnected'     self.chstate()  def execmsg(self):     'Translate the received message'     pass  def chstate(self):     'Change the state of the pipe depending on current state'     if self.state=='noconnected':         win32pipe.ConnectNamedPipe(self.hpipe,self.over)         self.state='connectwait'         return -6      elif self.state=='connectwait':         j,self.strbuf=win32file.ReadFile(self.hpipe,self.buffer,self.over)         self.state='readwait'         return -6      elif self.state=='readwait':         size=win32file.GetOverlappedResult(self.hpipe,self.over,1)         self.msg=self.strbuf[:size]         ret=self.execmsg()         self.state = 'noconnected'         win32pipe.DisconnectNamedPipe(self.hpipe)         return ret