Question

I have written a C++ class for Windows and Linux that creates a memory-mapped view for an file of arbitrary size n. The code for the class constructor can be seen here. I am currently testing the code on Windows 32 bit XP.

I have found for file sizes 0 < n <= 1.7GB , the constructor returns a valid pointer to a memory-mapped view. However, for a file size >= 2 GB, MapViewOfFile returns a NULL value and an error code of 8, “Not enough storage is available to process this command”. Evidently, Windows cannot find an available address space of size 2 GB in the process.

Therefore, I may need to modify the class constructor to create a set of smaller memory-mapped views totaling >= 2GB bytes && < 2 ^ 32 – 1 bytes. The other requirement is to create a mapping between each of the smaller memory-mapped views and a randomly accessed address in the process’ address space.

Previously, I used the following code for random access:

char* KeyArray;

try {
    mmapFile = new cMemoryMappedFile(n);
}
catch (cException e)
{
    throw;
}

KeyArray = (char *)(mmapFile->GetPointer());
KeyArray[i] = ...

How should I modify the class to handle these requirements?

Answer 1

I can’t see your pastebin link, but I can suggest a simple solution with a c++ class declaration. I think the implementation should be obvious from the comments:

class ShiftingMemMap
{
public:
    // constructs a dynamically shifting memory map of a file...
    ShiftingMemMap ( const char* fileName, size_t view_size = 4096 );

    // retrieve/set a byte at the given file offset. If the offset is not currently in-view,
    // shift the view to encompass the offset. The reference should not be stored for later
    // access because the view may need to shift again...
    byte& operator [] ( unsigned int_64_t offset );

private:
    int_64_t current_offset;
    size_t current_size;
};

All that being said, you could write a class that returns multiple views of a file to allow saving a reference for later and also editing different parts of the file simultaneously without having to shift the view back and forth repeatedly.

class MemMap
{
public:
    MemMap ( const char* filename );

    shared_ptr<MemMapView> View ( unsigned int_64_t offset, size_t size = 4096 );
};

class MemMapView
{
public:
    char& operator[] ( size_t offset );
};