I have a new program that will be generating a lot of Base64 encoded audio and image data. This data will be served via HTTP in the form of XML and the Base64 data will be inline. These files will most likely break 20MB and higher. Would it be more efficient to serve these files directly from the filesystem or would it be feasible to store the data in a MySQL database? Caching will be set up but overall unnecessary because it is likely that this data will be purged shortly after it is created and served.
i know that storing binary data in the DB is frowned upon in most circumstances but since this will all be character data I want to see what the consensus is. As of now, I am leaning toward storing them in the filesystem for efficiency reasons but if it is feasible to store them in a database it would be much easier to manage the data.
You need to weigh several factors:
Speed of access If you dump all your files in the filesystem, then some filesystems break or slow way down when you have too many files. I have seen many different directory segmentation schemes to address this issue.
Compression MySQL can easily compress your stored data. With a filesystem it may be harder to compress data. Using any kind of compression compression also raises a concern with the CPU speed on your server.
Backup/Restore A DB is made for easy backup/restore, how easy or hard it is for a filesystem varies greatly. If you have data in the filesystem to back/restore then that makes your backup/restore more complicated, and complicated is not good in a crisis.
Data loss tolerance With a DB you manipulate a record and all data is there in the same place. If you save your data to a file, you need to manipulate the associated file with extra code, and this this introduces another place to have bugs and possible data loss. More specifically, if you store part of the record in the filesystem and the rest in the DB, you need to go to great lengths to maintain transactional integrity. Nobody cares about this until you lose data, so pay attention to this aspect even though it may not seem important.
Replication It is easier to deal with replication concerns if you only have to do it at one level. Storing your data only in the DB and then replicating only the DB is far easier than doing that for the DB and also for the file system.
Cumbersomeness It is just more cumbersome to deal with larger DB backups than it is with smaller DB backups.
Those are the top issues that come to mind. I tried not to advocate for either solution, it really depends heavily on which file system you are talking about as well, and since that is not specified, you need to make that decision yourself.