Question

There is an A set of elements indexed [0, n]. At any time at most m elements from an A set can be active (in use), with an obvious condition that 0 <= m <= n. I would like to index those active elements inside a local array. An element can be deactivated dynamically while the program is executing and I would prefer that its index could be reused, when new elements are activated.

I would like to map an element index to it’s local index in the most efficient way as I’m using the local index for fast lookup of active element data.

Possible solutions of trivial hash function (element index == local index) and brute forcing through an associative list do not scale well with large n.

Dynamic expansion/shrinking of data structure is an obvious plus.

Thank you

Answer 1

I would like to map an element index to it’s local index in the most efficient way

There ain’t no such thing as the fastest code (tanstatfc) – Michael Abrash

There are many solutions to your problem.
First step is to choose a datastructure and a hash function.

Datastructure can be:

1. Plain array + straight hash
2. Array holding starting pointer to linked list + hash linking into the starting element
3. Binary tree where the hash denotes the branches of the tree
4. I’m going to stop there.

1 Plain array

This is the simplest solution. And if your allocations are blobby (that means they are clustered together in space) this might even be a good solution.
Here’s how it works:

1. You claim a big chunk of virtual memory. You can claim 2GB of memory (even on a 1GB machine), because it’s just virtual it will only get committed if you actually use it.
2. Split the block up in 4KB blocks, or multiples thereof (x86 processors use 4KB blocks) and make an index array start specify if a 4K block has been committed.
3. If you need to write, check in the index if the page has been committed, if it hasn’t, commit it.
4. Write to the list.
5. If you need to read, check the index, if the page hasn’t been committed, there is no hit, return false, else read the entry.

You can fit 2GB / 4bytes per entry = 500 million entries in this structure.
This works best for data that grouped in clusters that are close together.
If the indexes are random, this will be inefficient.

Here’s Delphi pseudo code:

Example code for straight list using Virtual memory

type
  TElement = record
    Data: string; //really a pointer to string in Delphi
    function PutInList(IndexNr: integer): PElement;
    constructor CreateFromList(Index: integer);
  end;

  PElement = ^TElement;
  TElements = array[0..0] of TElement;
  PElements = ^TElements;

const
  ArraySize = (1024 * 1024 * 1024 * 2); //2GB
  BlockSize = 4096;
  NumberOfBlocks = (ArraySize) div (BlockSize); //2GB in 4KB blocks
  BitsPerInt32 = 32;
  IndexCount = NumberOfBlocks div BitsPerInt32;

var
  IndexBlock: array[0..IndexCount-1]; //1 bit for each block = 64KB.

var
  Elements: PElements;

function ClaimVirtualBlock: PElements; 
begin
  FillChar(IndexBlock, #0, SizeOf(IndexBlock)); //Zero init indexblock
  Result:= PElements(VirtualAlloc(nil, ArraySize, MEM_RESERVE, PAGE_READWRITE));
end;

function GetAddress(Index: integer): PElement; inline;
var
  DestAddress: PElement;
  BlockIndex: Integer;
  BlockDWord: integer;
  BlockMask: integer;
  BlockNotAllocated: boolean;
begin
  //Create a new element in our list
  Integer(DestAddress):= Index * SizeOf(TElement); 
  BlockIndex:= Integer(DestAddress) div 4096;
  BlockMask:= 1 shl (BlockIndex mod 32);
  BlockIndex:= BlockIndex div 32;
  BlockNotAllocated:= (IndexBlock[BlockIndex] and BlockMask) <> 0;
  if BlockNotAllocated then begin
    IndexBlock[BlockIndex]:= IndexBlock[BlockIndex] or BlockMask;
    if VirtualAlloc(DestAddress, BlockSize, MEM_COMMIT, PAGE_READWRITE) = nil then
      raise EOutOfMemoryError.Create('Out of physical memory');
  end;
  Result:= DestAddress;
end;


function TElements.PutInList(IndexNr: integer): PElement;
begin
  Result:= GetAddress(IndexNr);
  Result^.Data:= Self.Data;
end;

constructor TElements.CreateFromList(Index: integer);
var
  ListElement: PElement;
begin
  ListElement:= GetAddress(Index);
  Self.IndexNr:= Index;
  Self.Data:= ListElement.Data;
end;

2 Array with linked list

1. Create an array with pointers to a linked list.
2. Hash the index, this points to your array item.
3. Walk through the linked list until you find the correct item.
4. For inserting an item: do step 1 and 2, and insert your item at the start.

This works best for data that has a very random index, with little change of colliding.
The success depends critically upon your hash function, it needs to select a different array entry as much as possible, too many collisions and you will just be walking the same linked list all the time.

Example code for array of linked lists

type  
  PElement = ^TElement;
  TElement = record
    Index: integer;
    Data: string;
    Next: PElement;
    procedure PutInList;
    constructor CreateFromList(AIndex: integer);
  end;

const
  LargePrimeNumber = 100003;

var
  StartArray: array[0..LargePrimeNumber-1] of PElement;

procedure InitList;
begin
  FillChar(StartArray, #0, SizeOf(StartArray));
end;

function IndexToArrayHash(AnIndex: integer): integer; inline;
begin
  Result:= AnIndex mod LargePrimeNumber;
end;

procedure TElement.PutInList;
var
  ArrayIndex: integer;
begin
  ArrayIndex:= IndexToArrayHash(Self.Index);
  Self.Next:= StartArray[ArrayIndex];
  StartArray[ArrayIndex]:= @Self;
end;

constructor CreateFromList(AIndex: integer);  
var
  ArrayIndex: integer;
  Element: PElement;
begin
  ArrayIndex:= IndexToArrayHash(AIndex);
  Element:= StartArray[ArrayIndex];
  while (Element <> nil) and (Element.Index <> AIndex) do begin
    Element:= Element^.Next;
  end; {while}
  if (Element <> nil) then begin
    Self.Index:= Element^.Index;
    Self.Data:= Element^.Data;
    Self.Next:= nil;
  end;
end;

3 binary tree using the bit in the index to traverse the tree

1. Create an empty tree with just a root
2. If we have an item to insert, use the bits in the index to traverse the tree (0 = left branch, 1 = right branch).
3. Whilst traversing the tree append higher ranked indexes below and insert lower ranked indexes above higher ones. (Heavy items sink to the bottom).

Example code using a binary tree

type
  PElement = ^TElement;
  TElement = record
    Left, Right: PElement;
    Index: integer;
    Data: string;
    procedure PutInList;
  end;

function GetFromList(AIndex: integer): PElement;

var
  Root: TElement;

const
  GoLeft: false;
  GoRight: true;

procedure InitRoot;
begin
  FillChar(Root, #0, SizeOf(Root));
end;

function TElement.PutInlist;
var
  CurrentElement: PElement;
  PrevElement:= PElement;
  Depth: integer;
  Direction: boolean;
begin
  PrevElement:= nil;
  CurrentElement:= @Root;
  Depth:= 0;
  //Walk the tree
  while (CurrentElement <> nil) and (CurrentElement.Index < Index) do begin
    PrevElement:= CurrentElement;
    Direction:= Odd(Index shr Depth);
    case Direction of
      GoLeft: CurrentElement:= CurrentElement^.Right;
      GoRight: CurrentElement:= CurrentElement^.Left;
    end; {case}
    Inc(Depth);
  end; {while}

  //Insert the item here
  case Direction of 
    GoLeft: PrevItem^.Left:= @Self;
    GoRight: PrevItem.Right:= @Self;
  end;

  //What to do with the currentItem.
  if Assigned(CurrentItem) then begin
    Direction:= Odd(CurrentItem^.Index shr Depth);
    case Direction of
      GoLeft: begin
        Self.Left:= CurrentItem;
        Self.Right:= CurrentItem^.Right;
      end; {GoLeft}
      GoRight: begin
        Self.Right:= CurrentItem;
        Left:= CurrentItem^.Left;
      end; {GoRight}
    end; {case}
  end; {if}
end;

function TElement.GetFromList(AIndex: integer): PElement;
var
  CurrentElement: PElement;
  Depth: integer;
  Direction: boolean;
begin
  CurrentElement:= @Root;
  Depth:= 0;
  //Walk the tree
  while (CurrentElement <> nil) and (CurrentElement.Index < Index) do begin
    Direction:= Odd(Index shr Depth);
    case Direction of
      GoLeft: CurrentElement:= CurrentElement^.Right;
      GoRight: CurrentElement:= CurrentElement^.Left;
    end; {case}
    Inc(Depth);
  end; {while}
  if Assigned(CurrentElement) and (CurrentElement^.Index = AIndex) then begin
    Result:= CurrentElement;
  end
  else Result:= nil;
end;

Recommend reading:
Knuth: TAOCP I: fundamental algorithms, chapter 2 ISBN 0-201-03809-9
Cormen, Leiserson & Rivest: Introduction to Algorithms, chapter III Data structures ISBN 0-07-013143-0