Question

I am working on a web application and i need to store images on server (these uploaded by user). i am generating new Guid for every new image.And i am creating three images of one image i.e. Original,icon and Thumb. still i am using three different folders to save these.My Project manager ask me why i haven’t create different folders for every users.
So my question is which is better way to store images between the following:

1). Upload images in only three folders i.e. Original, Icon and Thumb,
or
2). create folder for every user and then create three folders in it for original,thumb and icon.
one thing i would like to ask if the number of users will more than 100000 then will it effects performance???

which is better approach. please help me to take decision. Thanks

Answer 1

I’d store images by user. However, given that you potentially have a lot of users, it may not be that simple.

User file uploads are just that: user-specific file uploads. When you manage these files you often need to apply various procedures per-user. For example: 1) remove user account and delete all related files, 2) calculate amount of space a user uses, 3) list all files a user has uploaded, etc.

If you scatter these files across various directories it is much harder to efficiently implement the above procedures.

You specified that you may have more than 100.000 users. Depending on what your filesystem is, you may end up having troubles because of this. For instance in ext3, there is maximum of 32K subdirectories per directory, which can be a problem for organizing the files in directories per user (see: How many files can I put in a directory?).

Assuming that we cannot store more than 32K files or directories inside a directory then you would need to find out a way to work around this limit. For instance, you can split user folders into several subdirectories according to the first letter in username (and add an extra subdir for all the other starting characters):

users
   a
      aaron
         ...
   b       
      bertie
         ...
   ...
   misc
      _foobar 
         ...
      @jack
         ...

Now you only have roughly 100000/25=4000 users per directory, which is under the given 32K.

Thinking outside of the box, it may not be the right choice to store the images as files in a flat filesystem. There are database systems that are specifically created for storing and handling large numbers of files. Take MongoDB’s GridFS for instance. It is very efficient and scales to large number of files, also taking care of all the lower-level issues like correct filesystem usage. In MS SQL there is the FILESTREAM Storage, which is specifically good at storing files on an NTFS filesystem.