We have to store approximately 13 million files and perform some standard operations on the files. We are using Windows. The first question is can we store it on the filesystem? The total file size would be about 6000 TB. I am checking gridfs on MongoDB. I don’t know if this is a good approach. We will be using Java at the serverside.
If it can store these files comfortably, what would be the performance for fetching the file and serving it to the user and operations like file rename, metadata update etc.
We also need to backup all the files to a secondary filesystem storage later. But the files in gridfs would be stored in chunks. So the question is how can we quickly get all those files and send it to secondary filesystem.
Please let me know the approach I should take.
We have to store approximately 13 million files and perform some standard operations on
Share
MongoDB is web scale, so it should be fine