Question

Does it matter how many files I keep in a single directory? If so, how many files in a directory is too many, and what are the impacts of having too many files? (This is on a Linux server.)

Background: I have a photo album website, and every image uploaded is renamed to an 8-hex-digit id (say, a58f375c.jpg). This is to avoid filename conflicts (if lots of ‘IMG0001.JPG’ files are uploaded, for example). The original filename and any useful metadata is stored in a database. Right now, I have somewhere around 1500 files in the images directory. This makes listing the files in the directory (through FTP or SSH client) take a few seconds. But I can’t see that it has any effect other than that. In particular, there doesn’t seem to be any impact on how quickly an image file is served to the user.

I’ve thought about reducing the number of images by making 16 subdirectories: 0-9 and a-f. Then I’d move the images into the subdirectories based on what the first hex digit of the filename was. But I’m not sure that there’s any reason to do so except for the occasional listing of the directory through FTP/SSH.

Answer 1

FAT32:

• Maximum number of files: 268,173,300
• Maximum number of files per directory: 2¹⁶ – 1 (65,535)
• Maximum file size: 2 GiB – 1 without LFS, 4 GiB – 1 with

NTFS:

• Maximum number of files: 2³² – 1 (4,294,967,295)

• Maximum file size
  • Implementation: 2⁴⁴ – 2⁶ bytes (16 TiB – 64 KiB)
  • Theoretical: 2⁶⁴ – 2⁶ bytes (16 EiB – 64 KiB)
• Maximum volume size
  • Implementation: 2³² – 1 clusters (256 TiB – 64 KiB)
  • Theoretical: 2⁶⁴ – 1 clusters (1 YiB – 64 KiB)

ext2:

• Maximum number of files: 10¹⁸
• Maximum number of files per directory: ~1.3 × 10²⁰ (performance issues past 10,000)
• Maximum file size
  • 16 GiB (block size of 1 KiB)
  • 256 GiB (block size of 2 KiB)
  • 2 TiB (block size of 4 KiB)
  • 2 TiB (block size of 8 KiB)
• Maximum volume size
  • 4 TiB (block size of 1 KiB)
  • 8 TiB (block size of 2 KiB)
  • 16 TiB (block size of 4 KiB)
  • 32 TiB (block size of 8 KiB)

ext3:

• Maximum number of files: min(volumeSize / 2¹³, numberOfBlocks)

• Maximum file size: same as ext2
• Maximum volume size: same as ext2

ext4:

• Maximum number of files: 2³² – 1 (4,294,967,295)
• Maximum number of files per directory: 10 million approximately (can be extended with large_dir feature)
• Maximum file size: 2⁴⁴ – 1 bytes (16 TiB – 1)
• Maximum volume size: 2⁴⁸ – 1 bytes (256 TiB – 1)