Question

Background: I have a large 2D array of integers that I need to load into memory in PHP for each Apache request. I want it to take up less memory.

PHP stores ints in PHP_INT_SIZE bytes, which is 32 bits on most systems. All the integers are less than 2^16, which means they could be a short int (eg in C). Am I right in thinking that storing ints as short would take up half the RAM?

Ideally I’d like to be able to do:

$s = (short) 1234; // takes up 2 bytes instead of 4

More info:

• The array takes up about 100mb of RAM and is generated by including a 30MB var_export() dump
• The array is written in a cron process. Only the reading needs to be memory efficient (and quick)
• The only operations I need to do on the integers are comparing all of them (<, >, ===) and then reading a few of them (similar to the Floyd-Warshall algorithm)
• Reading each value from a DB is way too slow as there are a few hundred million reads per request

Some crazy ideas:

• Use pack() / unpack() but that would still store the values as 32 bit ints when they were unpacked
• Store the values as pixels in an image and use PHP’s GD library to read them (would this be slow)
• Use shmop_read() and have the Apache processes share the array
• Memcached might work but I have no experience with it and I guess it would be many times slower than a native PHP array
• Learn C++ and write a PHP extension
• Recompile PHP (or HipHop?) to use 2 bytes for ints
• Use Igbinary (useful, but will have same problem as pack())

Answer 1

I would not recommend last approach. 🙂

For the quick solution, I would pack 2 your integers in 1 PHP integer using this:

$big = $int1 + ($int2<<16);

And uppack as:

$int1 = $big & 65535;
$int2 = ($big>>16) & 65535;

Also, BIG thumbs up for using shared memory. This will make your APP way faster.