What are three main drawbacks of storing text data in one large ASCII text file as compared to splitting it up to multiple files? If I do split up, could you suggest a heuristic to choose a threshold size?
Share
Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
What are three main drawbacks of storing text data in one large ASCII text file as compared to splitting it up to multiple files? If I do split up, could you suggest a heuristic to choose a threshold size?
Typically loading the file into an editor is the big problem so if the use case for the file involves a user opening it up then its good to split it up.
For example when logging exceptions to a file I usually limit the size of the log file to 50 MB, because I know most text editors with current typical hardware can handle that pretty easily. This will change of course change with OS and hardware improvements. So 20 years from now mutli-GB text files might load nicely.
If you transferring data on the other hand the limits should be tuned to the transport mechanism. For example many email systems cap attachments around 10 MB.