Question

I have a server where archived files will be uploaded and stored. In order to improve performance of listing and checking file existance, I need to organzie the files in performance oriented directory structure.

I have a database table that will hold the real name of the files and their temp name.

###################################
FILES
###################################
id int auto_increment primary key,
name varchar (255),
temp_name varchar (255)

The root directory can contain a max of 1000 subdrectories ranging from 0-999. Each of the directories will hold 1000 files.

So the result will be

root/0 ==> will hold file having the id range from 1-999
root/1 ==> will hold file having the id range from 1000-1999
root/2 ==> will hold file having the id range from 2000-1999
.
.
.
root/999 ==> will hold file having the id range from 999,000-999,999

the directory where a file is stored can be found using the following equation

$directory = floor($file_id_from_db/1000);

WHERE $file_id_from_db is retrived from the files database table files.id.

The problem takes place when 1,000,000th file is uplaoded then I have to start storing the files in a second level.

I have to create a second level of directories ranging from 0 – 999 in the 0th directory
root/0/0 – root/0/999

then when I reach root/0/999 and I have placed 1000 files in it I need to move to
root/1/999 and so on till I reach root/999/999.

my current function looks something like that

function getPath($id){    
     $result = floor($file_id/1000);
     //Second level checks (Tried and crashed and burned)
     return "/$result"; 
}

I have no idea how to implement the logic for creating the subdirectories?

Thank for any suggestions.

Answer 1

If another approach is possible you can try the following:

• pad the db-id to e.g. 20 places (something that will suffice for the long run) like: 00000000000001665765
• split that string into chunks of suitalbe lenghts (e.g. 2) and rejoin those with your DIRECTORY_SEPARATOR

This will then be your storage path, e.g.:

$id = 1665765;
$paddedId = str_pad($id, 20, '0', STR_PAD_LEFT);

echo $path = '/' . implode(DIRECTORY_SEPARATOR, str_split($paddedId, 2));

// ==> /00/00/00/00/00/00/01/66/57/65

this way, you’ll have at most 100 directories/files per directory. (this will of course be different if you choose another split-length)

To easily create the directory you can use the third parameter for mkdir.

mkdir(dirname($path), 0755, true);