Question

Which is the better practice to store file? Directly store the file in database or just the location to that file?

Answer 1

It depends. You need to consider several things.

1. If you have a mickey mouse freeware database, meaning that it does not handle blobs appropriately (reads the blobs on every SELECT; does not store the blobs in a separate physical structure to the row; very slow with blobs; etc)

   • keep the files outside, store only the location
   • manually deal with the syncing of row.location vs the file system

2. If you have an enterprise SQL Platform, it is no problem at all to keep the blobs inside the database. In fact, retrieval is faster. These do not read the blobs on every SELECT, they are stored in a separate physical structure to the rows. The one extra read to get the blob if the SELECT requests it, is not a "performance problem".

3. The PAGESIZE in genuine SQL databases can be set as 2k; 4k; 8k; or 16k.

   • 2k is perfect for OLTP (small rows, small Transactions: you do not want to move 8K on every IO operation)
   • larger sizes are relevant based on how much OLAP you cater for
     • in your case, the average size of the files
     • there will be some waste in the unused portion of the last page, per row/blob.

4. The disadvantage of keeping the blobs in the database is, your database backups will be significantly larger.

   • Some enterprise databases (eg. SAP/Sybase) recognise that a page has not changed, and excludes it from the incremental backups
   • others have no incremental database backups.

5. The advantage of keeping the blobs in the database is:

   • data and referential integrity. You will not have the problem of having the rows that are out of synch with the blobs

   • the blobs are included in the backup: otherwise, upon a restore, the task of syncing the restored database with the restored files is a major problem.

   • I completed an assignment last year, where the customer had 130GB of data in the db, and 700GB of documents stored outside the db. After ten years of problems, they bit the bullet, and moved the documents into the db.
     Guess what, what was supposed to be a simple job (long but simple, because the references were supposed to be absolutely correct), ended up being massive, because there were so many (a) duplicates, and (b) invalid references.
     The resulting database was 630GB, there were 100GB of dupes. 2K pagesize.

Responses to Comments

1. Slash or Backslash
   Easy.

   • In the database, store slash only.
   • You need a way of identifying the target system, and an IsWindoze indicator. It should be higher up in the table hierarchy, not at the level where the Filename is located.
   • Whenever you report or display the Filename column, if IsWindoze, change the slashes to backslashes.

   You will have a similar problem with the DriveLetter and colon D:, which Unix does not have. Allow it only if IsWindoze.