I have built a USB data logger stick that shows up as a USB mass storage device when plugged in for file download. The files are recorded offline, but the stick can return limited ‘live’ data (device clock, battery charge, instantaneous sensor value) when a small (<512 bytes, 1 disk sector) special file on the device is read – basically like a UNIX device file.
The application that reads and uses/displays the logger data is written in Python.
I thought I was a clever fellow, until I went to actually read live data from this file. It turns out that under Windows (tested in XP and Win7 so far), only the first read from the application actually reads from the device – attempting to re-read the file (sector) returs stale cached data from the initial read, even 10+ minutes after the initial read, even if the file is closed and re-opened, or if the sector is written to and then re-read. Nor does accessing the file data via mmap() cause fresh reads. The subsequent read requests simply don’t make it to the device at all. I guess Windows ‘knows’ it has not modified the file itself, so the file contents could not possibly have changed.
Is there any sane way to force a subsequent read (preferably via a ‘standard’ Python file I/O call) to actually return a fresh read from the device rather than cached data?
To be clear, I am specifically referring to thwarting a READ cache, not syncing the disk contents after a write.
Does anything like this exist that is accessible from within Python?
Invalidate OS disk read cache
Using the win32 extensions for Python should allow you to access the CreateFile call directly to open the file. You may be able to use that to read your file.
Here are the Win32 extensions.