Basically I have some sort of third party software running on Windows 7 which does not take full advantage of installed RAM and keeps reading data from disk. As the whole content fits into memory, still leaving plenty to operate, the idea here is to pre-read all content with a self-written program to prime filesystem cache and preferably lock it into physical memory. Using a ramdisk would be too cumbersome.
Also, reading the content should occur in background while already running the third party software.
Some keywords come to mind:
- memory mapping.
- files larger than 4 GB.
What (free) programming language would you recommend for this task? Any idea on how to lock the mapped file into physical memory?
Since about every language deeper than scripting had mmap support, I went with Python for a first proof-of-concept. It worked out very well. Can easily keep a 12 GB dataset consisting of tens of thousands of files in pagecache, increasing performance of the original app.