Question

I’ve used C# to solve the following requirement..
– create an app the can receive a lot of data fast
– you must be able to analyse the received data while more are incoming.
– use as little CPU and disk as possible

My idea for an algorithm was..

SIZE = 10MB
Create a mmf with the size of SIZE
On data recived:
  if data can't fit mmf: increase mmf.size by SIZE
  write the data to mmf

-> The size on the disc are increased in chuncks of 10MB when the previous “room/space” are used.

How is the “increase mmf.size by SIZE” done in C#? I have found a lot of simple examples on creating mmfs and views but the only place (link) I have seen code that acutally increases the mmfs area uses code that can’t compile. Any help will be greatly appriciated.

EDIT
This causes an exception :

private void IncreaseFileSize()
{
    int theNewMax = this.currentMax + INCREMENT_SIZE;
    this.currentMax = theNewMax;

    this.mmf.Dispose();

    this.mmf = MemoryMappedFile.CreateFromFile(this.FileName, FileMode.Create, "MyMMF", theNewMax);
    this.view = mmf.CreateViewAccessor(0, theNewMax);            
}

This exception is thrown : The process cannot access the file ‘C:\Users\moberg\Documents\data.bin’ because it is being used by another process.

Answer 1

Once you map a file in memory, you cannot increase its size. This is a known limitation of memory mapped files.

…you must calculate or estimate the size of the finished file because file mapping objects are static in size; once created, their size cannot be increased or decreased.

One strategy would be to use chunks stored in non-persisted memory mapped files of a given size, say 1GB or 2GB. You would manage these through a top level ViewAccessor of your own design (probably doing basic passthru of the methods you need from the MemoryMappedViewAccessor).

Edit: or you could just create a non-persisted memory mapped file of a maximal size you expect to use (say 8GB to start, with a parameter to tune it on start-up of your application) and retrieve MemoryMappedViewAccessor‘s per logical chunk. The non-persisted file will not use physical resources until each view is requested.