Question

I am using PHP to handle some large files (around 500MB to 2 GB) where I need to write to small sections in unpredictable positions in the file. Is there a way to map the file to virtual memory so I do not have to make a fwrite() or fread() call? The major caveat is that it will be reading small chunks, but will be accessing a large amount of data in total, all within one file.

Answer 1

KingCrunch nails it on the head.. fwrite, fread and fseek really are the way to go..

The only reason I can think of why you may feel that you don’t want to use them, is perhaps that they are a bit harder to use in PHP?

If this is the case, why not create a simple wrapper class that calls these methods internally..

Edit based on comment

With virtual memory, I understood the windows term, which is memory mapped to the filesystem. If you are concerned about performance, and you are fine placing the entire file into memory instead, there’s two routes I can think off right now:

1. Create a tmpfs mount, and place the file there.
2. Use the php://memory stream, and place the entire file in there with stream_copy_to_stream.

You can also just do a file_get_contents, which also gives you the entire stream in memory, but I can definitely imagine that both tmpfs and php://memory will give you better performance and memory efficiency. I would personally go for the tmpfs mount.