Question

Is there a reason that opening a device file (rather than a regular file) using numpy’s memmap shouldn’t work?

self.surface = np.memmap('/dev/fb1', dtype=np.uint16, mode='r+', shape=(320,240))

I’m working with a custom kernel module that adds a framebuffer device, which works fine with python’s regular mmap module. But using numpy seems to hang the kernel’s mutex on accessing the filesystem or something (I’m really not sure exactly what’s happening).

My question here is specifically is this something that numpy’s memmap can’t handle and I should go a different way?

I’ve asked another question on unix stackexchange, but I feel like it’s 2 different questions so I’ve posted them both.

Obviously this is on linux (kubuntu maverick with custom kernel module)

Update:

Well, it turns out I can create the memmap fine. The problem it seems is that when I close the process without specifically closing the memmap object and it will just hang on the mutex in the kernel.

I have no idea if this issue is with numpy, or my kernel module, or somewhere else.

Answer 1

If your code works fine with the python mmap module, you can use it directly instead of numpy.memmap:

>>> fd = os.open("a", os.O_RDWR)
>>> buffer = mmap.mmap(fd, 0)
>>> surface = np.ndarray((320,240), np.uint16, buffer)

This has the other advantage that you have more control over the memory mapping used.

Now, python’s mmap has its own peculiarities. As the source shows, it calls msync on delallocation. Maybe this is where your program hangs? (You might be able to reproduce your issue with buffer.flush(), which also calls msync). Your solution of calling close() first probably works because it circumvents msync!